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I.  INTRODUCTION 


...the  highest  form  of  leadership  is  to  thwart  the  enemy's  plans.... Therefore  the  skillful  leader  subdues  the 
enemy's  troops  without  any  fighting.  He  captures  their  cities  without  laying  siege  to  them.  He  overthrozos 
their  kingdom  without  lengthy  operations  in  the  field.  With  his  forces  intact  he  disputes  mastery  of  the 
empire ,  and  ivithout  losing  a  man ,  his  triumph  is  complete.  The  weapon ,  not  being  blunted  by  use,  remains 
sharp  and  perfect. 

This  is  the  method  of  attacking  by  stratagem,  (p.  19  -20) 

What  enables  the  wise  sovereign  and  good  general  to  strike  and  conquer,  achieving  things 
beyond  the  reach  of  ordinary  men,  is  foreknowledge 

Sun  Tzu  (1966  adaptation  by  Rudnicki) 

The  principal  factor  altering  the  nature  of  the  threat  posed  by  the  modem  IADS  is  the  treatment 
of  data  and  information.  As  a  direct  descendent  of  the  technology  revolution,  the  introduction  of 
information  and  data-processing  networks  has  transformed  the  modem  IADS.  (p.  163) 

JSEAD  must  evolve  beyond  the  classic  "destroy  and  jam"  strategies  of  resource-rich  times  when 
US  SEAD  technology ,  personnel  training,  and  firepower  outstripped  virtually  any  opponent.  In  adapting 
to  the  IADS's  transition  from  shooter-based  to  information-based  orientations,  JSEAD  must  focus 
increasingly  on  information  denial  as  its  primary  task.  The  unwitting  transition  from  a  shooter-based  to 
an  information-based  IADS  provides  modem  SEAD  strategy  with  the  key  vulnerability  of  the  modem 
IADS  structure:  paralyze  the  enemy  by  denying  the  ability  to  acquire,  process,  or  transmit  data.  (p.  167) 

Brungess  (1997) 

American  military  officers  today  most  often  refer  to  Clausewitz's  unified  concept  of  a  general  friction 
(Gesamtbegriff  einer  allgemeinen  Eriktion)  as  the  "fog  and  friction"  of  war.  The  diverse  difficulties  and 
impediments  to  the  effective  use  of  military  force  that  those  possessing  military  experience  instinctively 
associate  with  this  phrase  are  generally  acknowledged  to  have  played  significant  roles  in  most ,  if  not  all,  of 
the  wars  since  Clausewitz's  time.  Even  in  a  conflict  as  inundated  with  technically  advanced  weaponry  as 
the  1991  Persian  Gulf  War  ( Operation  Desert  Storm),  there  was  no  shortage  of  friction  at  any  level, 
tactical,  operational,  strategic,  or  even  political.  Indeed,  close  examination  of  Desert  Storm  suggests  that 
frictional  impediments  experienced  by  the  winning  side  were  not  appreciably  different  in  scope  or 
magnitude  than  they  were  for  the  Germans  during  their  lightning  conquest  of  Prance  and  the  Low 
Countries  in  May  1940.  (p.  1) 

Watts  (1996) 

The  proceedings  of  the  board  convened  by  the  U.S.  Navy  to  investigate  the  (Iranian  airliner) 
incident  show  little  sign  of  any  primary  role  for  'bubble' formation  aboard  Aegis  cruisers  (United  States 
Department  of  Defense,  1988).  Instead,  the  role  of  the  multiple  technical  systems,  which  heretofore  served 
primarily  to  feed  information  to  the  tactical  operations  officer  seem  to  have  been  reversed.  Aboard  the 
Vincennes,  the  Aegis-directed,  computer-operated  anti-aircraft  missile  system  is  the  primary  system ,  with 
the  ship's  captain  primarily  involved  in  deciding  whether  to  activate  its  firing  mode  or  not.  In  this 
situation,  the  operator  has  become  an  input,  not  an  integrator,  and  the  captain  an  authorizor  rather  than  a 
commander  of  action.  Fordism  has  come  to  the  Navy,  one  of  the  last  repositories  of  craft  skills  and  the 
apprenticeship  method  in  the  U.S. 

Rochlin  (1993) 
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The  quotations  on  the  preceding  page  are  included  as  evidence  that  the 
art  of  war  has  always  depended  on  (and  will  continue  to  depend  on)  the  ability 
to  manage  information.  Some  have  speculated  that  advances  in  surveillance  and 
information  technologies  will  lift  the  "fog  of  war"  and  allow  "forces  total 
command  of  future  battlefields"  (Admiral  Owens,  cited  in  Inside  the  Navy,  23 
January  1995,  p.  3.  Referenced  by  Watts,  1996).  However,  Watts  (1996)  presents 
a  convincing  argument  that  "the  prospects  for  eliminating  friction  entirely 
appear  quite  dim  because  friction  gives  every  evidence  of  being  a  built-in  or 
structural  feature  of  combat  processes"  (p.2).  Watts  (1996)  further  argues  that 
"the  roots  of  surprise"  are  found  "in  circumstances  that  effect  honest,  dedicated, 
and  intelligent  men.  The  possibility  of  such  surprise  at  any  time  lies  in  the 
conditions  of  human  perception  and  stems  from  uncertainties  so  basic  that  they 
are  not  likely  to  be  eliminated,  though  they  might  be  reduced"  (p.  46).  Watts' 
argument  is  consistent  with  observations  of  the  impact  of  information 
technologies  in  other  fields  (e.g.,  Perrow,  1984).  Generalizing  from  these 
observations,  Klein  (1997)  concludes  that  any  increase  in  clarity  brought  about  by 
advances  in  information  technologies  will  be  counteracted  by  pressure  to 
"expedite  decision  cycles."  The  result  will  be  a  "situation  where  commanders 
are  making  faster  decisions,  at  the  same  level  of  uncertainty  that  they  were  used 
to"  (p.  15).  Rochlin  (1993),  argues  further,  that  friction  is  a  desirable  attribute. 

He  argues  that  friction  may  be  essential  to  stability  in  complex  organizations 
because  it  can  "give  an  organization  embarking  on  an  incorrect  or  errant  course 
the  opportunity  to  review  and  possibly  to  recover  before  major  problems  occur" 
(p.  198).  Rochlin's  analysis  emphasizes  the  fact  that  friction  has  both  negative 
and  positive  attributes.  The  checks  and  balances  that  resist  the  implementation 
of  a  "good"  decision  may  be  essential  for  protecting  the  system  against  "bad" 
decisions.  Friction  may  be  an  essential  attribute  for  a  high  reliability 
organization.  Thus,  this  report  starts  from  the  assumption  that  uncertainty  and 
the  requirement  to  process  information  to  extract  meaning  have  always  been  and 
will  remain  fundamental  to  the  art  of  war. 

Despite  the  argument  that  managing  information  has  always  been 
central  to  the  art  of  war,  there  is  a  pervasive  feeling  (e.g.,  Whitaker  &  Kuperman, 
1996)  that  we  are  in  the  midst  of  a  paradigm  shift  or  "revolution  in  military 
affairs"  (RMA)  driven  by  developments  in  information  technology  (including 
advances  in  sensors,  displays,  communications,  and  computing  devices). 
Brungess's  (1994)  analysis  of  developments  in  Integrated  Air.  Defense  Systems 
(IADS)  and  the  implications  for  the  suppression  of  enemy  air  defenses  (SEAD) 
mission  provides  a  good  example  of  how  information  and  communication 
technologies  are  changing  modem  warfare.  This  feeling  of  a  paradigm  shift  has 
generated  the  construct  "information  warfare"  (IW)  and  this  construct  has 


become  the  banner  for  focused  efforts  of  the  Air  Force  and  other  branches  of  the 
Department  of  Defense  (DoD)  to  get  out  in  front  of  the  new  technology  and  its 
application  in  warfare.  This  effort  reflects  increasing  awareness  that  the  masters 
of  the  new  technologies  will  be  the  masters  of  information  and  that  the  masters 
of  information  will  be  the  victors  in  future  conflicts  (as  they  have  always  been  in 
the  past). 

The  challenge  to  get  out  in  front  of  advances  in  communications  and 
information  technologies  is  not  unique  to  the  domain  of  war.  This  challenge  is 
pervasive  in  modem  society.  It  is  also  being  faced  by  the  transportation 
industry,  manufacturing,  energy  production,  chemical  process  control, 
education,  etc.  Virtually,  every  aspect  of  life  is  touched  in  some  way  by  these 
advances.  In  response  to  this  challenge,  a  new  discipline  is  emerging — cognitive 
systems  engineering  (CSE).  The  target  phenomenon  for  this  new  discipline  is  the 
application  and  design  integration  of  information  technology  to  facilitate  work. 
Work  is  defined  quite  broadly  to  include  any  focused  effort  to  move  through  a 
problem  space  (i.e.,  work  domain)  to  achieve  specific  objectives.  Information 
technology  is  also  defined  broadly  to  include  both  artificial  (e.g.,  computers)  and 
natural  (e.g.,  human  experts)  information  processing  systems. 

The  development  of  a  new  discipline  reflects  a  growing  sense  that  no 
existing  paradigm  (e.g.,  information  science,  control  theory,  psychology, 
industrial  engineering,  etc.)  spans  the  phenomenon  of  work;  further,  the  pieces 
defined  by  various  existing  paradigms  do  not  appear  to  add  up  to  a 
comprehensive  understanding  of  the  phenomenon.  Stated  simply,  the  problem 
of  work  has  emergent  properties  that  can  not  be  completely  understood  within 
the  current  disciplinary  frameworks.  Extrapolations  from  these  disciplines  fail  to 
predict  the  impact  of  new  technologies  (e.g.,  the  paperless  office)  and  fail  to 
provide  a  foundation  for  design  decisions. 

The  development  of  a  new  discipline  also  reflects  an  increasing 
appreciation  of  the  limits  of  a  reductionist  paradigm  in  science  and  the 
assumption  of  linearity  upon  which  that  paradigm  is  based  (e.g.,  Gleik,  1987; 
Waldrop,  1992).  The  problem  of  work  is  increasingly  being  cast  within  the 
context  of  nonlinear  dynamics  (Flach,  1997).  This  new  appreciation  is  reflected  in 
a  changing  value  system.  In  reductionist  paradigms,  control  and  internal 
validity  were  paramount.  In  the  emerging  paradigm,  ecological  validity  takes 
precedence.  Thus,  science  is  moving  out  of  the  laboratories  to  engage  problems 
in  their  natural  ecology.  Rather  than  looking  to  experimental  physics  as  the 
model  for  investigation,  scientists  in  the  new  discipline  are  increasingly  looking 
to  fields  such  as  ethology  and  anthropology  (e.g.,  Hutchins,  1995;  Klein,  Orasanu, 
Calderwood,  &  Zsambok,  1993)  for  their  methods  and  models. 
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This  report  has  two  complementary  objectives.  The  first  objective  is  to 
introduce  the  new  field  of  CSE  to  those  who  are  wrestling  with  the  problems  of 
IW.  The  second  objective  is  to  begin  exploring  the  IW  problem  as  an  opportunity 
to  test  the  CSE  paradigm.  The  next  section  will  give  a  brief  introduction  to  CSE. 
Then  the  IW  problem  will  be  framed  in  the  context  of  the  emerging  paradigm. 
This  will  help  to  introduce  the  IW  problem  and  will  provide  a  more  concrete 
view  of  CSE.  The  final  section  will  lay  out  a  proposal  for  how  the  CSE  paradigm 

might  be  applied  to  the  problem  of  designing  the  tools  that  will  lead  to  victory  in 
the  new  millennium.  J 
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II.  COGNITIVE  SYSTEMS  ENGINEERING:  AN  HISTORICAL 

PERSPECTIVE 


Many  developments  in  science  and  engineering  have  contributed  to 
shaping  the  field  of  CSE.  Even  a  dedicated  attempt  to  write  the  history  of  the 
field  would  be  incomplete.  However,  even  an  incomplete  perspective  on  some 
of  the  historical  factors  that  have  shaped  the  field  will  help  readers  to  better 
appreciate  why  CSE  has  evolved  as  a  discipline.  Therefore,  I  will  begin  the 
section  with  a  brief  (and  biased)  review  of  what  I  consider  to  be  some  important 
factors  that  have  shaped  CSE. 

One  important  root  of  CSE  was  the  work  of  Taylor  and  Gilbreth  on 
"Scientific  Management"  (e.g.,  see  Konz,  1987).  This  work  was  inspired  by  the 
technologies  that  enabled  mass  production  (as  well  as  by  the  economic 
opportunities  offered).  The  key  contribution  of  these  men  was  the  importance 
that  they  gave  to  measurement  and  standardization.  First,  they  helped  to 
establish  "time"  or  "rate"  (units/time)  as  a  fundamental  ruler  for  judging  the 
"best  way"  or  "the  "right  way."  Armed  with  this  ruler,  they  had  a  basis  for 
controlled  comparisons  by  which  the  best  way  to  shovel  coal  or  the  best  way  to 
lay  bricks  could  be  empirically  determined.  Further,  these  controlled 
experiments  provided  a  basis  for  shifting  the  responsibility  for  choosing  the 
"work  method"  from  the  individual  worker  or  artisan  to  the  manager.  It  became 
management's  responsibility  to  provide  resources  needed  to  determine  the  best 
way  and  to  provide  the  training  and  incentives  required  to  get  the  workers  to 
comply.  Also,  assembly  lines  needed  to  be  designed.  Because  of  the  number  of 
people  involved,  the  time  and  money  were  not  available  to  let  assembly  lines 
evolve  to  a  stable  configuration.  To  a  large  extent,  the  stability  had  to  be 
designed  into  the  system  (the  line  needed  to  be  "balanced")  if  it  were  to  function 
properly.  Thus,  the  time  of  component  operations  needed  to  be  predictable. 

This  led  to  increasing  standardization  of  methods  and  tools.  Thus,  the  industrial 
engineer's  or  efficiency  expert's  job  was  to  determine  the  "best  way"  through 
empirical  tests  on  component  operations;  to  organize  those  components  into  a 
balanced  configuration  to  produce  safe,  efficient,  and  stable  processes;  and  to 
implement  training  and  incentive  programs  so  that  workers  would  comply  with 
the  standards  upon  which  the  design  was  developed. 

In  addition  to  the  introduction  of  empirical  methods,  an  enduring  theme 
that  can  be  traced  to  Taylor  and  Gilbreth  is  that  increased  effort  (working  harder) 
was  not  the  only  path  to  increased  productivity.  For  example,  Taylor  showed 
that  it  was  possible  to  increase  productivity  by  mandating  rest  breaks.  People 
began  to  recognize  that  it  was  possible  to  increase  productivity  by  "working 
smarter." 
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Rochlin  (1993)  presents  an  interesting  discussion  of  some  of  the  negative 
side  effects  of  the  Scientific  Management  approach.  These  side  effects  arise  from 
the  conflict  between  efficiency  (which  often  depends  on  limiting  friction)  and 
reliability  (which  often  benefits  from  increased  friction  in  terms  of  checks  and 
safe  guards).  Negative  side  effects  from  the  Scientific  Management  approach 
include  de-skilling  workers  and  disenfranchising  workers  from  a  sense  of 
ownership  or  responsibility  for  the  quality  of  the  product  as  a  whole. 

Whereas,  the  prime  mover  for  Taylor  and  Gilbreth  was  efficiency.  World 
War  II  (and  the  technology  of  war)  and,  later,  the  development  of  nuclear  power 
plants,  caused  increased  concern  about  issues  associated  with  safety.  Efforts  to 
determine  the  "right  way"  shifted  from  an  emphasis  on  "time"  to  an  increased 
concern  about  "human  error"  as  a  "cause"  of  potentially  catastrophic  system 
failures.  The  Human  Engineering  Division  within  the  Air  Force's  Armstrong 
Laboratory  was  an  early  center  for  studying  human  performance  in  the  aviation 
domain.  Fitts  and  Jones'  analyses  of  pilot  error  in  operating  aircraft  controls 
(1947a)  and  of  reading  and  interpreting  aircraft  instruments  (1947b);  and 
Christensen's  (1947)  analysis  of  navigation  were  early  examples  of  studies  of 
cognition  in  the  wilds  of  aviation.  Fitts  and  Jones'  analyses  were  based  on 
retrospective  verbal  reports  from  pilots.  Analyses  of  the  errors  reported  by 
pilots  inspired  laboratory  studies  of  such  things  as  shape  coding  of  controls 
(Jenkins,  1947)  and  alternative  designs  for  altimeter  displays  (Grether,  1949)  that 
eventually  led  to  design  innovations.  Christensen's  (1947)  field  observations  of 
navigators  earned  him  membership  in  the  Pole  Vaulter's  Club  as  the  first  civilian 
to  fly  over  the  North  Pole  in  an  Air  Force  aircraft.  These  field  observations  were 
followed  up  with  experimental  evaluations  (1949)  that  led  to  the  design  of 
improved  plotters  that  became  the  Air  Force  standard. 

Similar  work  was  being  done  in  Great  Britain  by  researchers  such  as 
Bartlett  and  Craik  who  studied  the  effects  of  stress  and  fatigue  on  skilled 
performance  in  the  "Cambridge  cockpit"  (designed  around  a  Spitfire  cockpit). 
The  Cambridge  program  made  many  important  "contributions  in  the  areas  of 
aircrew  selection  and  training,  the  effects  of  sleep  loss  and  fatigue,  and  various 
aspects  of  visual  perception  and  display  design"  (Edwards,  1988,  p.  7).  The 
combination  of  field  observations  with  empirical  laboratory  studies  set  the  flavor 
for  "human  engineering"  analyses  to  come.  However,  the  balance  between  field 
observations  and  empirical  evaluations  achieved  in  this  early  work  is  a  standard 
that  is  rarely  achieved  today. 

The  work  of  Fitts  and  Bartlett  (and  their  colleagues)  is  often  recognized 
as  a  critical  step  toward  the  development  of  the  formal  disciplines  of  human 
factors  (engineering  psychology  or  ergonomics).  Many  people  currently  active 
in  the  field  proudly  trace  their  intellectual  roots  to  either  Fitts  or  Bartlett.  Several 
important  themes  emerged  from  this  era.  First,  the  work  in  these  laboratories 
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further  demonstrated  the  promise  of  an  empirical,  experimental  program  for 
investigating  the  "best  way."  Second,  the  emphasis  on  achieving  the  "best  way" 
shifted  from  the  managers  of  the  work  to  the  designers  of  the  tools  (technology). 
It  became  clear  that  the  probability  of  human  error  was  influenced  by  the  design 
of  tools.  Finally,  the  most  important  theme  was  that  the  errors  were  often  traced 
to  failures  to  process  information.  There  was  an  increased  appreciation  for  the 
mental  or  information  processing  aspects  of  work  (e.g.,  perception,  memory,  and 
decision  making). 

Research  on  human  error  in  nuclear  power  got  great  impetus  as  a  result 
of  the  accident  at  Three  Mile  Island.  Early  work  (e.g..  Technique  for  Human 
Error  Rate  Prediction  [THERP])  characterized  human  error  in  terms  of  error 
probabilities  and  analyzed  the  work  environment  in  terms  of  performance 
shaping  factors  (PSFs)  that  could  potentially  increase  or  decrease  the  probability 
of  human  error.  These  PSFs  reflected  general  design  principles  (e.g.,  consistency 
in  the  layout  of  instruments)  that  had  their  origin  in  the  early  "human 
engineering"  analyses  of  Fitts  and  his  colleagues. 


However,  because  of  the  high  degrees  of  automation  in  nuclear  power 
production,  attention  began  to  shift  from  the  perception-/ action-based  errors 
(slips)  that  dominated  Fitts  and  Jones's  analyses,  to  errors  of  intention  (mistakes). 
In  the  automated  world  of  process  control,  the  operator  was  less  a  real  time 
controller  and  more  a  supervisor— whose  principal  function  was  to  detect  and 
diagnose  faults.  In  this  domain,  much  of  the  interesting  "activity"  was  below  the 
surface.  Thus,  it  was  difficult  to  make  inferences  about  the  factors  that 
influenced  performance  from  observing  activities  alone.  Also,  it  was  difficult  to 
characterize  the  problem  solving  processes  in  terms  of  probabilities.  Verbal 
protocol  analysis  became  a  critical  supplement  to  field  observations,  and 
qualitative  models  of  the  strategies  of  the  human  problem  solver  began  to 
replace  the  more  quantitative  chronometric  and  probability-based  analyses. 


RIS0  National  Laboratory  was  an  important  center  for  early  research  on 
human  fault  diagnosis  (Vicente  gives  a  brief  account  of  the  early  years  at  RIS0; 
1997).  Rasmussen  and  Jensen's  (1973;  see  also  Rasmussen,  1986;  Rasmussen, 
Pejtersen,  &  Goodstein,  1994)  field  study  of  electronic  trouble-shooting  was  an 
important  early  study.  It  relied  extensively  on  verbal  protocols.  These  studies 
were  conducted  in  the  electronics  shop  at  RIS0.  Audio  recordings  were  made  of 
technicians  as  they  diagnosed  problems  with  scientific  instruments  that  were 
brought  to  them  to  repair  (e.g.,  multichannel  analyzers,  oscilloscopes,  TV 
receivers,  etc.).  The  technicians  were  not  asked  to  introspect  about  their  thought 
processes,  but  were  asked  to  "merely  tell  what  they  were  trying  to  accomplish, 
what  they  were  doing  and/ or  observing,  and  with  which  instruments 
(Rasmussen  et  al.,  1994,  p.  86).  Analysis  of  the  protocols  allowed  multiple 
strategies  for  diagnosis  to  be  identified  and  described.  An  important  aspect  of 
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this  work  was  that  the  strategies  most  commonly  adopted  by  the  technicians 
(e.g.,  topographic  search)  were  not  the  ones  that  were  predicted  by  abstract 
analysis  of  problem  solving  (e.g.,  hypothesis  and  test).  Rasmussen  (1986) 
considers  whether  the  strategies  of  the  technicians  were  rational: 

The  experiment  discussed  here  clearly  indicates  that  the  task  is  defined  by  the 
technicians  primarily  as  a  search  to  find  where  the  fault  originates  in  the 
system.  They  are  faced  with  a  system  which  they  suppose  has  been  working 
properly,  and  they  are  searching  for  the  location  of  the  discrepancy  between 
normal  and  defective  states.  They  do  not  see  the  task  as  a  more  general 
problem-solving  task  in  order  to  understand  the  actual  functioning  of  the 
failed  system  and  thus  to  explain  why  the  system  has  the  observed  faulty 
response....  What  is  rational  depends  upon  the  performance  criteria  adopted 
by  the  person.  Normally  a  reasonable  criterion  for  a  maintenance  technician 
is  to  locate  the  fault  as  quickly  as  possible,  and  only  in  special  circumstances 
will  his  criterion  be  that  of  minimizing  the  number  of  measurements....  From 
this  point  of  view,  the  procedures  found  in  our  records  are  rational  since  in 
most  cases  the  faults  were  found  within  very  reasonable  times....  The  system 
designer  with  his  theoretical  background  may  quite  naturally  value  as  rational 
the  "elegant"  deductive  procedure  which  is  informationally  very  efficient  and 
based  upon  few  observations,  but  this  criterion  is  not  the  appropriate  one  on 
the  basis  of  which  performances  in  real  life  maintenance  work  can  be  judged. 

(p.  55-56) 

Despite  the  productivity  and  influence  of  the  work  at  RIS0,  much  of  the 
work  on  human  factors  and  human  machine  systems  tended  to  emulate  the 
experimental  methods  and  laboratory  paradigms  derived  from  the  work  of  Fitts 
and  the  Applied  Psychology  Unit  at  Cambridge.  Through  the  '70s  and  the  '80s 
there  was  a  heavy  emphasis  on  experimental  rigor.  Field  studies  such  as  the 
work  on  electronic  trouble  shooting  at  RIS0  were  exceptions,  rather  than  the 
rule.  Most  of  the  work  on  human  performance  and  cognition  utilized  abstract 
experimental  paradigms  (e.g.,  sine  wave  tracking,  probe  reaction  time  tasks, 
probability  judgments,  visual  search,  syllogisms,  etc.).  These  paradigms  were 
assumed  to  reflect  general,  fundamental  aspects  of  human  performance  and 
there  was  relatively  little  concern  about  representativeness.  Naturalistic  field 
studies  were  rarely  reported  in  the  technical  literature.  Chronometric  analyses 
dominated  the  literature  and  verbal  protocol  analyses  were  generally  regarded 
with  deep  skepticism. 

Attention  and  workload  became  a  central  theme  for  laboratory-based 
empirical  work  in  the  '70s  and  '80s.  The  problem  of  Three  Mile  Island  and, 
increasingly,  problems  in  aviation  and  other  domains  were  viewed  as  problems 
of  "too  much  data."  Measuring  "mental  workload"  became  a  critical  challenge 
for  researchers.  Important  design  questions  focused  on:  How  many  people  were 
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needed  to  meet  the  data  processing  demands  of  a  job  (e.g.,  is  a  single-operator 
attack  helicopter  feasible)?  How  can  automation  be  used  to  off-load  processing 
demands  from  the  human  operator  (e.g.,  can  the  copilot  be  replaced  by  an 
automated  pilots  associate)?  How  can  the  data  be  packaged  (integrated  in 
graphical  displays  or  distributed  over  modalities)  to  minimize  the  workload? 

In  the  mid  '80s,  in  part  stimulated  by  a  failure  to  achieve  consensus  on  a 
reliable  ruler  for  measuring  workload,  and  in  part  inspired  by  advances  in 
graphical  displays  and  successes  such  as  the  graphical  user  interface  on  personal 
computers,  the  characterization  of  the  problem  began  to  shift.  The  problem 
appeared  to  be  less  a  limitation  in  data  processing  and  more  a  problem  of 
understanding.  During  this  period,  it  became  increasingly  clear  that  the  capacity 
to  process  information  depended  critically  on  the  representation  of  that 
information.  Extracting  information  from  a  table  of  numbers  might  be 
overwhelming,  extracting  that  same  information  from  a  graph  might  be  virtually 
effortless.  Further,  it  was  not  simply  the  "form"  of  the  representation,  but  the 
correspondence  between  the  form  and  the  meaning  of  the  process  (work)  that 
was  being  represented  (e.g.,  Flach  &  Bennett,  1996;  Rasmussen  &  Vicente,  1989; 
Woods,  1991). 

In  the  aviation  domain,  this  shift  in  emphasis  from  data  processing  to 
understanding  was  evident  in  the  emergence  of  a  new  construct — situation 
awareness.  Here,  the  inspiration  came  from  observations  of  striking  differences 
in  how  individuals  responded  to  the  complex  information  processing  demands 
that  might  be  associated  with  combat.  One  individual  is  consistently 
overwhelmed  and  lost,  where  another  responds  instinctively  and  almost 
effortlessly.  Both  individuals  have  the  same  external  representation.  However, 
one  appears  to  have  a  superior  internal  representation — better  situation 
awareness.  In  other  words,  one  pilot  is  more  expert! 

During  this  period,  Klein  and  his  colleagues  (e.g.,  Klein,  Calderwood,  & 
Clinton-Cirocco,  1986)  began  studying  decision  making  in  natural  environments. 
They  began  by  "observing  and  taking  protocols  from  urban  fireground 
commanders  about  emergency  events  that  they  had  recently  handled"  (Klein, 
1993,  p.  139).  Klein  (1993)  discovered  that: 

The  fireground  commanders'  accounts  of  their  decision  making  do  not  fit  into 
a  decision-tree  framework.  The  fireground  commanders  argued  that  they  were 
not  "making  choices,"  "considering  alternatives ,"  or  "assessing 
probabilities."  They  saw  themselves  as  acting  and  reacting  on  the  basis  of 
prior  experience;  they  were  generating,  monitoring,  and  modifying  plans  to 
meet  the  needs  of  the  situations.  We  found  no  evidence  for  extensive  option 
generation.  Rarely  did  the  fireground  commanders  contrast  even  two  options. 

We  could  see  no  way  in  which  the  concept  of  optimal  choice  might  be  applied. 
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Moreover,  it  appeared  that  a  search  for  an  optimal  choice  could  stall  the 
fireground  commanders  long  enough  to  lose  control  of  the  operation 
altogether.  The  fireground  commanders  were  more  interested  in  finding 
actions  that  were  workable,  timely,  and  cost  effective,  (p.  139) 

As  a  result  of  observations  of  decision  making  in  natural  environments, 
Klein  developed  his  recognition-primed  decision  model”  which  emphasized  the 
importance  of  perception  and  the  "zeroing-in"  aspects  of  the  decision  problem. 
Equally  important,  however,  was  the  fact  that  Klein's  work  helped  to  stimulate 
wider  interest  in  the  problem  of  "expertise"  and  "naturalistic  decision  making" 
(e.g.,  Klein,  Orasanu,  Calderwood,  &  Zsambok,  1993)  and  helped  to  legitimize 
verbal  protocol  analysis  and  field  observations  as  essential  to  a  complete 
understanding  of  human  performance  in  complex  work  environments. 

In  1995,  Hutchins'  book  "Cognition  in  the  Wild"  was  published.  This 
book  summarized  a  comprehensive  field  (ethnograghic)  study  of  ship  navigation 
that  began  in  the  '80s.  Hutchins  (1995)  chose  the  title  "in  the  wild"  to  distinguish 
between  "the  laboratory,  where  cognition  is  studied  in  captivity,  and  the 
everyday  world,  where  human  cognition  adapts  to  its  natural  surroundings.  I 
hope  to  evoke  with  this  metaphor  a  sense  of  an  ecology  of  thinking  in  which 
human  cognition  interacts  with  an  environment  rich  in  organizing  resources"  (p. 
xiv).  Hutchins'  description  of  how  variations  in  map  design  shaped  the 
computation  problem,  making  some  problems  easier  and  others  harder,  provides 
a  clear  demonstration  of  the  fundamental  link  between  the  "right  way"  and  the 
"right  representation." 

The  realization  that  the  functionality  of  a  representation  (internal  or 
external)  relates  to  its  correspondence  to  problem  semantics  (to  a  situation)  has 
opened  up  the  problem  of  work  analysis.  Whereas,  the  emphasis  of  the 
information  processing  approach  was  on  internal  human  processing  constraints 
(channel  capacity,  bottlenecks,  or  resource  limitations),  there  is  now  increasing 
interest  in  constraints  within  the  situation.  There  is  growing  appreciation  that 
the  search  for  the  right  way '  has  to  be  framed  in  terms  of  both  an  analysis  of 
human  information  processing  (awareness)  and  an  analysis  of  the  semantics 
intrinsic  to  the  work  domain  (situation).  In  other  words,  the  problem  of  the 
"right  way"  involves  a  distributed  information  processing  system  that  spans  the 
human  and  the  work  ecology.  This  is  the  focus  of  CSE— to  understand  the 
semantics  of  situations  and  to  discover  how  awareness  can  be  facilitated  through 
the  engineering  of  representations.  These  representations  can  be  engineered 
through  training  (internal  representations)  or  through  the  design  of  visualization 
tools  (external  representations). 

The  works  of  Rasmussen,  Klein  and  Hutchins  also  help  to  show  that  it 
was  possible  for  researchers  to  discover  general  principles  and  broad  insights 
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into  human  cognition  by  delving  deeply  into  the  particulars  of  specific  work 
domains.  These  works  illustrate  that  the  image  of  human  cognition  that  was 
molded  from  general,  laboratory  paradigms  did  not  always  hold  up  in  the  wild. 
The  behavior  of  technicians,  fireground  commanders,  and  navigators  often  did 
not  conform  to  expectations  based  on  logical  analyses.  Further,  the  "nonsense" 
laboratory  tasks  derived  from  the  logical  analyses  of  information  processing 
tended  to  miss  the  essence  of  the  natural  tasks.  Semantic  and  contextual  aspects 
that  were  critical  to  performance  in  the  natural  world  tended  to  be  abstracted  out 
of  the  laboratory  tasks. 

In  sum,  the  safety  tradition  was  inspired  by  field  observations  of  natural 
phenomena  (e.g.,  accidents).  However,  early  researchers  (Fitts'  human 
engineering  group  and  Bartlett's  applied  psychology  unit)  were  extremely 
innovative  in  extracting  key  features  of  the  natural  phenomenon  that  could  be 
studied  in  the  laboratory,  using  rigorous  controls  and  quantitative  measures  of 
performance.  The  success  of  the  laboratory  paradigms  tended  to  overshadow 
the  phenomenon-based  insights  that  inspired  them.  As  a  result,  the  '70s  and  '80s 
were  dominated  by  a  paradigmatic  information  processing  approach  to  human 
performance  where  the  laboratory  paradigms  became  the  phenomena  of  interest. 
Field  studies  were  rare,  and  it  was  generally  assumed  that  natural  tasks  could  be 
understood  as  a  combination  of  the  laboratory  tasks  (e.g.,  landing  an  airplane 
might  be  considered  to  be  a  combination  of  tracking,  visual  search,  and  decision 
making  tasks). 

However,  pockets  of  phenomena-centered  research  were  maintained. 
Rasmussen,  Klein,  and  Hutchins  are  prominent  representatives  of  researchers 
who  were  not  seduced  by  the  power  of  laboratory  paradigms.  These  researchers 
maintained  a  phenomenon-centered  science.  In  part,  this  was  motivated  by  the 
practical  need  to  address  problems  in  natural  systems  like  nuclear  power  plants. 
But  it  was  far  more  than  this.  It  was  also  motivated  by  a  sense  that  human 
cognition  in  the  wild  was  fundamentally  different  than  cognition  observed  in 
captivity;  a  sense  that  the  context-free,  logical  puzzles  and  toy  worlds  of  the 
cognition  laboratory  were  not  representative  of  the  demands  faced  in  natural 
work  domains;  and  a  sense  that  performance  in  the  laboratory,  where  human 
limitations  were  easily  catalogued  and  the  fallibility  of  human  reasoning  seemed 
obvious,  was  not  representative  of  the  creative,  adaptive  nature  of  people  in  their 
fields  of  expertise.  Rasmussen,  Klein,  and  Hutchins  were  not  alone.  However, 
their  work  was  prominent  and  helped  draw  attention  to  earlier  insights  of  people 
such  as  deGroot  (e.g.,  Vicente  &  Wang,  1998),  Brunswik  (Hammond,  1993;  Kirlik, 
1995),  and  Gibson  (Flach,  Hancock,  Caird,  and  Vicente,  1995).  Gradually,  there 
seems  to  be  growing  appreciation  for  the  importance  of  ethnographic 
observations  for  a  basic  understanding  of  cognition.  Researchers  are  beginning 
to  consider  ways  to  scale  the  methods  of  observation  and  analysis  to  the 
complexity  of  the  natural  phenomenon,  as  opposed  to  cutting  the  phenomenon 
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into  pieces  that  fit  conveniently  into  laboratory  paradigms.  This  is  the  challenge 
that  has  stimulated  the  new  discipline  of  CSE — to  understand  the  holistic  and 

emergent  properties  of  work  in  a  way  that  will  generalize  to  design— to  achieve 
expertise  by  design. 

This  section  has  presented  CSE  in  an  historical  context.  Table  1 
summarizes  this  discussion  in  terms  of  four  "eras"  of  work  on  human-machine 
systems.  These  eras  are  not  necessarily  distinct  historical  epochs,  but  represent 
themes  that,  though  interleaved  in  time,  are  distinctive  in  terms  of  the 
technological  changes  that  are  addressed.  Each  era  has  framed  the  problem 
differently  and  each  era  has  focused  on  a  different  metric  for  comparing  and 
evaluating  potential  solutions. 


Table  1:  An  historical  context  for  approaches  to  human-machine 
systems 


ERA 

Key  Technologies 

Defining  Problem 

Measurement 

Scientific 

Management 

Assembly  Line 

Mass  Production 

Efficiency 

Standardization 

Line  Balancing 

Rate 

Pieces/Minute 

Human  Error 

Advanced  Military  Systems 
Nuclear  Power 

Reliability 
" Too  many  hitman 
errors " 

Design  Guidelines 

Error  Rate 

BHEP 

(basic  human  error 
probability) 

Information 

Overload 

Nuclear  Pozver 

Civil  Aviation 
Automated  Systems 

Attention 

( bottlenecks ,  limited 
resources) 

"Too  much  data" 

Mental  Workload 
(subjective,  physiological, 
primary  &  secondary 
task) 

Visualization 

Graphical  Interfaces 

Situation  Aiuareness 
"Too  little 
understanding" 
Representations 

Decision-Cycle  Time? 
Robustness? 

In  the  first  era,  the  motivating  technology  was  the  assembly  line.  The 
key  problems  were  efficiency,  standardization  of  methods,  and  line  balancing. 
The  primary  performance  metric  was  production  rate — how  many  units  per 
minute  could  be  produced.  With  the  advances  in  technology  arising  during 
World  War  II  and,  later,  with  the  development  of  nuclear  power,  focus  shifted 
from  efficiency  to  safety  or  reliability.  The  defining  issue  was  human  error  and 
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how  to  reduce  the  probability  of  system  failure.  The  primary  performance 
metric  was  error  probability.  However,  as  nuclear  process  research  continued 
and  as  other  systems  (like  civil  aviation)  became  more  complex,  the  focus  began 
to  shift  to  the  explosion  of  data  that  operators  were  being  asked  to  deal  with. 
Increasingly,  system  failures  were  being  attributed  to  limitations  of  human 
information  processing.  The  defining  issue  became  attention,  and  numerous 
metrics  (subjective,  physiological,  primary  task,  and  secondary  tasks)  for 
measuring  mental  workload  were  developed.  This  is  still  a  major  concern  in 
human  factors.  As  graphical  displays  became  available,  it  began  to  be  evident 
that,  with  the  appropriate  representation,  large  amounts  of  data  could  be 
processed  very  efficiently.  Thus,  the  focus  is  now  shifting  to  the  problem  of 
understanding  or  situation  awareness — how  to  integrate  complex  data  in  a  way 
that  will  facilitate  understanding.  This  is  the  era  of  data  visualization.  In  the 
following  sections  we  will  suggest  two  possible  metrics  for  the  visualization 
problem — decision-cycle  time  and  robustness  (or  adaptability). 

The  next  section  will  present  the  CSE  framework.  A  unique  feature  of  the 
CSE  framework  is  the  explicit  attempt  to  analyze  situations  at  multiple  levels  of 
abstraction  to  reveal  the  constraints  that  are  intrinsic  to  a  work  domain.  Section 
IV  will  begin  to  flesh  out  the  framework  in  the  context  of  IW.  Hopefully,  this 
exercise  will  provide  insights  about  both  CSE  and  IW.  The  final  section  will 
present  recommendations  for  a  research  program  to  explore  IW  using  the  CSE 
framework. 
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III.  THE  cognitive  systems  engineering  framework 


Science  and  technology  have  always  tried  to  answer  the  question  "how?" 

How  does  this  or  that  mechanism  work?  What  are  the  laws  and  causal 
properties  that  underlie  this  phenomenon  and,  in  the  case  of  technologies, 
how  can  such  knowledge  he  used  to  develop  a  useful  tool  or  machine? 

However,  science  and  technology  rarely,  if  ever,  address  the  question 
"Why?"  It  seems  to  be  outside  their  respective  universe  of  discourse.  The 
question  is  ruled  inadmissible  or  inappropriate  for  the  methods  and 
capabilities  at  hand.  I  reject  this  absolutely.  I  submit  that  questions  of 
how  and  why  are  so  mutually  dependent  that  they  should  never  be 
considered  independently,  and  I  attribute  much  of  our  present 
circumstance  to  this  unfortunate  and  unhappy  division.  Those  who  know 
how  must  always  ask  why.  Those  who  ask  why  must  always  think  how. 
(Hancock,  1996,  p.  461) 

One  of  the  first  considerations  for  an  analysis  framework  is  to  specify  the 
"unit"  of  analysis.  That  is,  what  are  the  "objects"  of  analysis?  Gilbreth 
introduced  the  "therblig"  as  a  fundamental  unit  for  motion  analysis.  Any 
manual  operation  could  be  characterized  in  terms  of  these  fundamental  units  of 
motion.  The  information  processing  approach  introduced  stages  of  information 
processing  to  characterize  qualitatively  different  mental  processes.  These  stages 
represented  fundamental  cognitive  operations,  and  tasks  were  described  in  terms 
of  the  relative  demands  on  these  stages  (e.g.,  perceptual  tasks,  memory  load, 
decision  tasks,  and  manual  tasks).  Early  work  on  human  error  characterized 
elemental  actions  (both  mental  and  manual)  in  terms  of  basic  human  error 
probabilities  (BHEPs).  An  assumption  of  all  these  approaches  was  that  there  was 
an  elemental  level  of  behavior  that  could  be  used  to  characterize  all  human 
interactions.  The  assumption  was  that  a  description  of  work  in  terms  of  its  basic 
behavioral  elements  would  allow  inferences  about  the  nature  of  system 
performance  (e.g.,  workload,  error  probability,  or  response  speed).  Further,  it 
has  been  assumed  that  studies  of  the  fundamental  units,  stripped  of  all 
extraneous  context,  could  provide  a  human  performance  database  that  would 
support  extrapolations  to  the  performance  of  human-machine  systems.  In  fact,  it 
has  generally  been  assumed  that  getting  rid  of  the  extraneous  context  was  a 
necessary  condition  in  order  to  achieve  broad  generality  of  inference. 

The  CSE  approach  takes  a  different  tack.  This  approach  begins  with  the 
assumption  that  context  is  not  extraneous,  but  fundamental  in  shaping  human 
performance.  Thus,  it  is  important  that  analyses  address  the  situartion  level 
constraints.  In  fact,  one  could  go  further  and  argue  that  because  of  the  adaptive 
nature  of  human  cognition,  the  situation  constraints  will  generally  dominate  for 
experienced  or  expert  operators  performing  in  the  domains  of  their  expertise. 
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The  context-free  laboratory  research  provides  important  information  about  the 
lower  thresholds  of  performance  and  may  generalize  well  in  predicting  the 
performance  of  novices  (or  experts  in  unfamiliar  contexts).  However,  it  is  very 
difficult  to  extrapolate  from  these  context-free  tasks  to  predict  the  performance 
limits  of  experts  in  their  natural  work  ecology.  The  literature  is  full  of  examples 
(such  as  the  observations  from  Rasmussen  and  Klein  cited  in  the  last  section) 
where  generalizations  from  the  laboratory  have  proven  to  be  very  naive  when 
tested  against  the  performance  of  experts  in  their  natural  work  environments. 

When  one  shifts  the  focus  from  the  human  information  processor  to  the 
situation,  one  quickly  discovers  that  there  is  no  single  atomistic  level  of 
description  that  is  privileged  (e.g.,  see  Marr,  1980;  Rasmussen,  1986;  Rasmussen 
et  al.,  1994).  As  Marr  and  Rasmussen  have  both  noted,  the  multiple  levels  of 
analysis  are  loosely  coupled.  There  must  be  a  mapping  from  one  level  to 
another,  but  there  are  also  emergent  properties  at  each  level.  That  is,  a 
description  at  one  level  of  abstraction  will  provide  insights  into  the  phenomenon 
that  are  not  possible  at  other  levels.  A  complete  understanding  requires  that  the 
analyses  address  multiple  levels  of  abstraction. 

Rasmussen  (1986)  has  characterized  the  multiple  levels  of  description  in 
terms  of  two  dimensions — level  of  abstraction  and  level  of  decomposition.  This 
framework  is  shown  in  Figure  1.  Rasmussen  (1986)  has  distinguished  five 
different  levels  for  characterizing  complex  socio-technical  systems  with  respect 
to  the  abstraction  dimension.  Brief  descriptions  are  presented  here.  However, 
these  levels  will  be  fleshed  out  more  fully  in  the  context  of  IW  in  later  sections  of 
this  report: 

Functional  Purpose.  This  level  identifies  the  goals  and  values  of  the  work 
domain.  What  is  the  purpose  of  the  system  and  what  are  the 
figures  of  merit  for  gauging  the  success  of  the  system? 

Abstract  Function.  This  level  specifies  the  problem  domain  in  terms  of 

abstract  dimensions  that  bridge  between  the  values  reflected  at  the 
level  of  functional  purpose  and  the  physical  constraints  that  bound 
the  flow  of  activities  and  the  operation  of  technologies  to  support 
those  goals. 

General  Function.  This  level  decomposes  the  system  into  processing  units 
with  identifiable  inputs  and  outputs.  Each  processing  unit  is 
characterized  in  terms  of  a  transfer  function  that  translates  inputs 
to  outputs. 

Physical  Function.  This  level  identifies  the  types  of  technology  (e.g.,  human 
operator,  computer,  hydraulic  or  mechanical  system,  etc.)  which  is 
associated  with  each  of  the  functions. 
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Physical  Form.  This  level  provides  a  detailed  accounting  of  the  space/time 
properties  of  the  system  (e.g.,  where  things  are  located,  what's 
connected  to  what,  etc.)  and  of  system  performance  (e.g.,  what 
happens,  when). 


AGGREGATION  -  DECOMPOSITION 
WHOLE  PART 


FUNCTIONAL 

PURPOSE 

ABSTRACT 

FUNCTION 

GENERAL 

FUNCTION 

PHYSICAL 

FUNCTION 

PHYSICAL 

FORM 

. 

Figure  1:  A  framework  for  analyzing  situations.  A  full  understanding 
of  a  situation  requires  descriptions  at  multiple  levels  of  abstraction 
and  at  different  levels  of  decomposition. 


Each  level  of  abstraction  refers  to  qualitatively  different  categories  of 
constraints  that  contribute  to  the  situation.  These  qualitatively  different 
categories  are  related  in  a  nested  hierarchical  structure.  Categories  at  higher 
levels  within  this  hierarchy  provide  the  rational  context  for  understanding  why 
lower  level  constraints  are  significant.  Categories  at  lower  levels  in  the  hierarchy 
provide  the  physical  details  of  hozv  constraints  at  higher  levels  can  be 
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implemented.  The  nested  hierarchy  is  typically  represented  as  a  triangle  (Figure 
2).  This  graphic  reflects  the  intuition  that  relatively  few  categories  are  needed  to 
meaningfully  characterize  high  level  constraint  of  the  work  domain.  However, 
for  lower  levels  many  more  categories  and  distinctions  are  relevant  to  a 
meaningful  characterization.  This  difference  in  level  of  decomposition  reflects 
the  second  dimension  of  analysis — part-whole  decomposition.  Together  these 
dimensions  create  the  two-dimensional  space  for  characterizing  the  analysis 
process  shown  in  Figure  1.  As  a  result  of  inquiry  into  many  different  work 
domains  and.  after  analyzing  many  verbal  protocols  of  domain  experts, 
Rasmussen  found  that  experts  tend  to  reason  along  the  diagonal  of  this  space. 
That  is,  expert  reasoning  tends  to  move  from  global  descriptions  of  functional 
goals  to  detailed  specifications  of  particular  actions  on  specific  components. 


Figure  2:  Rasmussen's  (1986)  abstraction  hierarchy  represents  a 
nested  hierarchy  in  which  relatively  few  categories  provide  a 
meaningful  characterization  of  the  functional  purpose  of  a  complex 
system,  but  many  more  categories  are  needed  to  meaningfully 
characterize  distinctions  in  physical  form. 

Figure  3  (from  Flach,  Eggleston,  Kuperman,  &  Dominguez,  1998)  shows 
how  different  types  of  classical  system  analyses  fit  within  the 
abstraction/ decomposition  space  for  understanding  the  problem.  Note  that 
these  different  analyses  fall  along  the  diagonal  of  the  analysis  space  and  thus  are 
consistent  with  the  pattern  seen  in  expert  reasoning.  The  hope  is  that  by 
integrating  information  represented  along  this  diagonal  we  can  begin  to  address 
questions  of  both  how  and  why. 

A  unique  attribute  of  the  CSE  approach  is  an  attempt  to  integrate  across 
the  component  analyses  to  achieve  a  global  understanding  of  the  distributed 
control  system.  Top  down  relations  along  the  diagonal  help  to  uncover  the 
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underlying  rationality  of  the  work  domain.  This  rationality  provides  the  reasons 
why  things  are  done  or  the  reasons  why  things  are  significant.  This  is  important 
to  understanding  the  focusing-in  problem  of  cognition.  That  is,  what  causes 
certain  features  to  pop  out  in  a  way  that  allows  experts  to  focus  in  on  critical 
aspects  of  a  problem  early  in  the  analysis.  In  other  words,  what  is  the  basis  for 
the  high  situation  awareness  achieved  by  experts?  How  is  it  that  they  can 
recognize  good  solutions  as  the  first  to  be  considered  (e.g.,  see  Klein,  1993)? 
Further,  how  can  representations  be  designed  to  facilitate  these  recognitional 
processes  and  to  support  high  levels  of  situation  awareness? 

Whole-Part  Decomposition 


Global  Elemental 


Figure  3:  An  illustration  of  how  different  aspects  of  a  cognitive  task 
analysis  fit  within  the  overall  analysis  space  defined  by  the  abstraction 
and  decomposition  dimensions.  Reasoning  down  the  diagonal  helps 
to  reveal  the  rationality  that  determines  why  things  are  done. 
Reasoning  up  the  diagonal  helps  to  reveal  the  causal  relations  that 
determine  how  things  are  done  (from  Flach,  Eggleston,  Kuperman,  & 
Dominguez,  1998) 


Bottom  up  relations  along  the  diagonal  help  to  uncover  the  causal 
structure  within  the  work  domain.  This  causal  structure  provides  a  production¬ 
like  description  of  how  things  can  be  accomplished  within  the  work  domain. 
This  kind  of  description  is  critical  for  the  design  of  rule-based,  counting-out 
processes  that  are  typical  of  classical  artificial  intelligence  and  expert  systems. 
These  relations  can  be  critical  in  the  design  of  automated  systems:  Automating 
the  well-behaved  (rule-based  or  routine)  aspects  of  the  work  can  free  up  human 
resources  to  be  focused  on  supervising  the  "messier"  aspects  of  the  work. 
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Within  the  CSE  approach  the  top  down  and  bottom  up  relations  are 
complementary  perspectives  that  contribute  to  a  complete  understanding  of  the 
workspace.  A  primary  goal  of  the  CSE  approach  is  to  make  the  links  along  this 
diagonal  explicit  (as  opposed  to  an  implicit  understanding  encapsulated  within  a 
particular  analyst's  experience).  Unlike  classical  and  modem  (e.g.,  optimal) 
approaches  to  control  systems  that  begin  with  restrictive  assumptions  (e.g., 
linearity,  a  quadratic  cost  function,  etc.)  in  order  to  build  quantitatively  powerful 
models  of  the  system,  CSE  focuses  on  the  assumptions,  themselves.  Classical 
and  optimal  control  models  make  assumptions  that  turn  messy  problems  into 
well-defined  problems  that  allow  quantitative  solutions.  But  CSE  is  all  about  the 
"messiness,"  because  that  is  why  operators  are  included  in  most  control 
systems — to  deal  with  the  messiness  (the  unanticipated  variability).  If  the 
problems  were  well-defined  and  met  the  limiting  assumptions  of  classical  or 
modem  control  models,  then  there  would  be  no  need  for  the  human.  The  goal  of 
cognitive  engineering  is  to  understand  the  messiness  as  the  most  interesting  part 
of  the  problem.  As  a  result,  the  models  of  CSE  tend  to  rely  more  on  the 
qualitative  insights  of  control  theory,  rather  than  on  the  quantitative  methods. 
This  does  not  mean  that  quantitative  models  are  not  used  or  that  they  are  not 
valuable  tools  for  developing  qualitative  insights.  But  the  quantitative  models 
are  not  sufficient.  In  many  respects,  CSE  is  most  important  for  exploring  regions 
just  outside  the  limits  of  a  quantitative  control  analysis  of  the  problem.  These  are 
the  regions  where  automated  solutions  are  likely  to  break  down.  These  are  the 
regions  where  human  expertise  is  an  essential  ingredient  for  success.  In  this 
respect,  CSE  is  a  means  of  providing  a  qualitative  glimpse  over  the  horizon  set 
by  limits  of  our  quantitative  control  models.  It  is  for  this  reason  that  the  CSE 
approach  is  particularly  well-suited  for  addressing  the  messy  problems  of  "fog 
and  friction"  that  are  at  the  heart  of  IW. 

From  a  design  perspective,  CSE  generally  focuses  on  the  nature  of 
problem  representations  as  a  means  for  dealing  with  messy  problems.  Through 
the  design  of  interfaces,  CSE  attempts  to  buildextemal  representations  that 
allow  operators  to  directly  perceive  significant  features  of  a  problem  (e.g.,  see 
Bennett,  Nagy,  &  Flach,  1997;  Flach  &  Bennett,  1996;  Rasmussen  &  Vicente,  1989; 
Vicente,  1992;  Woods,  1991).  Through  the  design  of  training  systems  and 
protocols,  CSE  helps  to  attune  internal  representations  (in  Rasmussen's  [1986] 
terms,  the  dynamic  internal  model)  to  significant  features  of  the  problem  (e  g 
Flach,  Lintem,  &  Larish,  1990;  Flach,  1997).  It  is  too  early  in  the  analysis  process 
to  speculate  about  the  details  of  interfaces  or  training  programs  for  IW. 

However,  the  final  section  of  this  report  will  speculate  on  the  directions  that 
research  and  analysis  might  take  that  will  eventually  provide  a  basis  for 
advancing  our  ability  to  manage  the  friction  and  fog  of  war  in  the'context  of 
current  and  future  advances  in  information  technologies. 
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IV.  INFORMATION  WARFARE:  A  COGNITIVE  ENGINEERING 

PERSPECTIVE 


This  section  will  be  organized  around  the  different  levels  of  abstraction. 
At  each  level  different  aspects  of  the  IW  problem  will  be  discussed. 

Fundamental  questions  at  each  level  will  concern  the  availability  of  descriptions 
at  the  appropriate  level  of  decomposition. 

Functional  Purpose 

The  fundamental  issue  at  this  level  of  abstraction  is,  why  does  the  system 
exist?  What  is  the  purpose?  The  first  place  to  look  for  answers  to  these  questions 
is  within  the  definition  of  IW.  One  definition  of  IW  is  "any  action  to  deny, 
exploit,  corrupt,  or  destroy  the  enemy's  information  and  its  functions;  protecting 
ourselves  against  those  actions;  and  exploiting  our  own  military  information 
functions"  (Widnall  &  Fogleman,  1995,  pp.  3-4;  cited  in  Whitaker  &  Kuperman, 
1996).  This  definition  explains  what  IW  is  in  terms  of  actions  (deny,  exploit,  etc.) 
and  the  object  of  those  actions  (information).  However,  the  definition  does  not 
address  the  issue  of  purpose.  What  are  the  functions  of  information?  Why  is 
control  of  information  desirable? 

The  answers  to  the  questions  above  cannot  be  found  by  focusing  on  the 
technology  (the  means  of  IW).  Rather,  focus  must  be  shifted  to  the  use  of  this 
technology.  What  is  the  information  used  for?  The  purpose  is  to  achieve  victory 
in  war.  Thus,  the  first  step  in  a  CSE  analysis  is  to  ask  what  the  information 
technology  is  used  for.  So,  in  this  section,  the  emphasis  will  be  on  the  ends  of 
war. 


The  means  of  war  may  change,  but  at  the  highest  level,  the  goals  and 
purposes  of  war  have  not  changed  much  since  the  time  of  Sun  Tzu.  At  a  global 
level,  the  availability  of  information  technologies  will  not  change  the  ultimate 
purposes  of  war.  Basic  aerospace  doctrine  (Department  of  the  Air  Force,  1992) 
outlines  nine  principles  of  war: 

Objective:  Direct  military  operations  toward  a  defined  and  attainable 
objective  that  contributes  to  strategic,  operational,  or  tactical  aims. 

Offensive:  Act  rather  than  react  and  dictate  the  time,  place,  purpose, 
scope,  intensity,  and  pace  of  operations.  The  initiative  must  be  seized, 
retained,  and  fully  exploited. 

Mass:  Concentrate  combat  power  at  the  decisive  time  and  place. 

Economy  of  Force:  Create  usable  mass  by  using  minimum  combat  power 
on  secondary  objectives.  Make  the  fullest  use  of  all  forces  available. 
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Maneuver  Place  the  enemy  in  a  position  of  disadvantage  through  the 
flexible  application  of  combat  power. 

Unity  of  Command:  Ensure  unity  of  effort  for  every  objective  under  one 
responsible  commander. 

Security:  Protect  friendly  forces  and  their  operations  from  enemy  actions 
that  could  provide  the  enemy  with  unexpected  advantage. 

Surprise:  Strike  the  enemy  at  a  time  or  place  or  in  a  manner  for  which  he 
is  unprepared. 

Simplicity:  Avoid  unnecessary  complexity  in  preparing,  planning,  and 
conducting  military  operations.  (Department  of  the  Air  Force,  1992,  Fig  1- 
l,p.l) 

Figure  4  shows  one  possible  network  of  interactions  among  these  nine 
principles  that  might  reflect  some  of  the  trade-offs  involved  in  IW.  This 
hierarchical  structure  does  not  represent  differences  in  priority.  Rather,  the 
hierarchy  highlights  functional  interdependencies  among  the  nine  principles. 
These  interdependencies  may  be  critical  for  evaluating  trade-offs  across 
competing  goals  within  IW. 

At  the  top  of  this  hierarchy  is  Unity  of  Command.  One  of  the 
fundamental  problems  with  advances  in  information  technology  is  the 
distributed  nature  of  information  and  the  requirement  to  coordinate  multiple 
distributed  actors.  The  challenge  is  to  maintain  a  unified  command  while 
supporting  distributed  execution;  the  key  is  for  the  distributed  elements  to 
function  as  a  coordinated  whole,  acting  toward  an  appropriate  goal.  In  Owens' 
(1995)  terms  the  challenge  is  to  get  "the  right  information  to  the  right  warfighter 
at  the  right  time."  Whitaker  and  Kuperman  (1996)  compare  the  problems  of  IW 
to  problems  of  computer-supported  cooperative  work  (CSCW)  and  group 
decision  support  systems  (GDSS)  in  general.  These  challenges  have  created  a 
need  for  a  shared  information  space — "...  a  medium  that  could  be  used 
dynamically  ...  to  help  people  to  share  their  view  of  the  world  with  others 
through  joint  manipulation  of  each  person's  personal  models  of  the  situation" 
(Bannon,  1989,  p.  13).  Rochlin,  LaPorte,  and  Roberts'  (1987)  contrasted  the 
formal  command  structure  on  an  aircraft  carrier  with  the  more  informal 
command  structure  that  evolved  (self-organized)  to  deal  with  the  distributed 
information  and  the  dynamic  demands  during  landing  on  an  aircraft  carrier. 

This  command  structure  appears  to  adapt  to  changing  information  and  changing 
loads.  Whitaker  and  Kuperman  (1996)  suggest  the  construct,  "Common 
Battlespace  Picture,"  to  characterize  a  central  representation  that  is  mutually 
accessible,  mutually  interpretable,  mutually  meaningful,  and  mutually 
manipulable.  The  communication  structures  on  aircraft  carriers  (as  described  by 
Rochlin  et  al.,  1987)  seem  to  have  evolved  naturally  so  that  a  Common 
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Battlespace  Picture  emerges  from  the  interactions  among  the  crew.  The 
challenge  of  IW  will  be  to  provide  the  information  links  to  facilitate  the 
development  of  such  a  shared  representation  among  joint  forces  over  a  much 
broader  battle  campaign.  Rochlin  (1993)  uses  the  naval  term  "having  the 
bubble": 

When  you  have  the  bubble,  all  of  the  charts,  the  radar  displays,  the 
information  from  the  console  operators,  and  the  inputs  from  others  and  from 
the  senior  staff fall  into  place  as  parts  of  a  large  coherent  picture.  Given  the 
large  amount  of  information,  and  the  critical  nature  of  the  task,  keeping  the 
bubble  is  a  considerable  strain.  On  many  U.S.  Navy  ships,  operations  officer 
shifts  are  held  to  no  more  than  two  hours.  'Losing  the  bubble'  is  a  serious  and 
ever-present  threat;  it  has  become  incorporated  into  the  general  conversation 
of  operators  as  representing  a  state  of  incomprehension  or  misunderstanding, 
even  in  an  ambiance  of  good  information. 

This  notional  'bubble'  is  one  of  the  key  elements  in  obtaining  high  and 
reliable  systemic  performance  in  organizational  settings  that  involve 
considerable  risk,  large  and  basically  uncontrollable  uncertainties,  and  a  high 
degree  of  organizational  as  well  as  technical  complexity,  operating  in  real  time 
situations  where  decisions  with  irrevocable  consequences  must  be  made 
quickly,  in  real  time,  on  the  spot  (Rochlin,  1989).  At  the  cognitive  level,  the 
conception  of  the  'bubble'  makes  it  possible  for  Navy  operations  officers,  air 
traffic  controllers,  and  (in  a  somewhat  different  context)  experienced  pilots  to 
integrate  a  system  of  vast  operational,  technical  and  cognitive  complexity  into 
a  single,  mental  representative  spatio-temporal  map  that  organizes  and  orders 
the  flow  of  information  and  allows  decisions  to  be  made  on  the  basis  of  overall, 
systemic  situations  and  requirements,  (p.  215) 

On  one  hand,  information  technology,  including  computer  networks  and 
the  use  of  the  electromagnetic  spectrum  for  sensing  and  communications,  has  led 
to  an  explosion  of  the  amount  of  information  available  and  has  contributed  to  the 
dispersion  of  that  information.  On  the  other  hand,  information  technology  offers 
important  opportunities  for  integrating  that  information  to  create  a  unified 
framework  for  making  command  decisions.  Thus,  depending  on  how  it  is  used, 
information  technology  can  exacerbate  the  problems  or  it  can  provide  unique 
solutions  to  the  Unity  of  Command  challenge. 

It  is  also  important  to  note  that,  as  Rochlin  et  al/s  (1987)  analysis  of  the 
aircraft  carrier  suggests.  Unity  of  Command  does  not  necessarily  mean 
centralized  command.  Watts  (1996)  points  to  economics  as  an  example  of  a  self¬ 
organizing  system.  He  quotes  Hayek  (1988),  who  writes  that  "modem 
economics  explains  how  such  an  extended  order  can  come  into  being,  and  how 
it,  itself,  constitutes  an  information-gathering  process,  able  to  call  up,  and  to  put 
to  use,  widely  dispersed  information  that  no  central  planning  agency,  let  alone 


22 


any  individual,  could  know  as  a  whole,  possess,  or  control"  (p.  14).  This  is  one  of 
the  major  challenges  of  IW — to  design  command  and  control  structures  that 
maximize  the  effectiveness  of  information  technologies  to  achieve  coordinated 
and  effective  action. 


Figure  4:  The  principles  of  war  related  to  three  fundamental  aspects 
of  IW — command  &  control,  counterinformation,  and  information 
assurance. 


In  order  to  achieve  Unity  of  Command  in  the  face  of  the  complexities  of 
advanced  distributed  information  systems,  it  will  be  increasingly- important  to 
cut  through  the  fog  and  friction  of  war  to  focus  on  the  crux  of  the  problem.  That 
is.  Simplicity  will  be  at  a  premium.  Klein's  (1997)  group  has  found  that  in 
Marine  field  headquarters  and  in  nuclear  power  plant  control  rooms  there  were 
many  more  people  than  necessary.  These  excess  people  added  to  the  complexity 
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of  the  problem  unnecessarily.  They  were  a  source  of  noise  and  increased 
computational  load.  It  was  difficult  to  identify  the  key  decision  makers  so  that 
they  could  be  provided  the  "right  information"  at  the  "right  time."  In  both  cases, 
efficiency  and  effectiveness  of  operations  were  increased  by  eliminating  people 
from  the  control  rooms.  The  lesson  here  is  that  more  data  does  not  necessarily 
require  more  data  handlers.  Added  agents  can  also  add  to  the  information 
burden,  without  facilitating  the  decision  process.  Sometimes,  less  is  more. 

Hand  in  hand  with  Simplicity  is  the  need  for  clearly  defined  Objectives. 
More  than  anything  else,  it  was  a  failure  to  maintain  clearly  defined  objectives 
that  hindered  operations  in  Vietnam.  Clearly  defined  objectives  will  be  very 
important,  so  that  distributed  cooperating  agents  don't  end  up  working  at  cross¬ 
purposes.  These  objectives  (or  shared  intentions)  will  help  provide  a  context  in 
which  it  will  be  easier  to  interpret  and  anticipate  local  adaptations  among 
cooperating  agents.  To  a  large  extent,  the  meaning  of  an  act  can  only  be  judged 
relative  to  the  intentions.  If  the  intentions  are  vague  and  ill-defined,  then  the 
meaning  of  actions  will  be  ambiguous  and  coordination  will  be  difficult  to 
achieve.  Clearly  defined  objectives  help  to  focus  attention  and  resources  on  the 
crux  of  the  problem. 

Economy  of  Force  has  become  an  increasingly  important  concern  since  the 
end  of  the  Cold  War  and  the  break-up  of  the  Soviet  Union.  It  is  becoming 
increasingly  difficult  to  justify  military  expenditures,  and  the  joint  forces  are 
being  challenged  to  do  more  with  less.  Information  technologies  offer  at  least 
three  obvious  paths  to  increased  economy.  First,  automation  can  reduce  the 
number  of  humans  required  to  operate  some  systems.  Crew  sizes  have  been 
reduced  on  aircraft  and  on  ships  as  a  result  of  advances  in  automation.  Second, 
automation  can  reduce  the  exposure  of  humans  to  hazardous  situations  (e.g., 
Uninhabited  Aerial  Vehicles  [UAVs]  and  other  remotely  controlled  vehicles). 

This  can  significantly  reduce  costs  associated  with  attrition.  A  third  source  of 
economy  expected  from  the  increased  application  of  information  technology  is 
the  ability  to  adapt  through  software,  rather  than  hardware.  That  is,  in  the  past, 
special  missions  have  required  specially  engineered  systems  (e.g.,  the  "Wild 
Weasel"  for  attacking  SAMs  [surface-to-air-missiles]).  However,  with  the 
addition  of  "smart"  systems,  a  single  platform  may  be  able  to  adapt  to  a  wider 
variety  of  applications.  Thus,  for  example,  a  few  aircraft  platforms  may  be 
adapted  to  a  wide  range  of  missions  through  software  modifications.  This 
translates  into  economies  of  scale  for  production  and  also  significantly  reduces 
maintenance  costs. 

Simple,  clearly  defined,  affordable  Objectives  provide  the  basis  for  a 
unified  command  and  control  structure  that  is  capable  of  proactively  dictating 
the  time,  place,  purpose,  scope,  intensity,  and  pace  of  operations.  This  Offensive 
posture  contributes  to  (and  derives  from)  our  ability  to  Surprise  the  enemy,  to 
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Maneuver  effectively,  to  Mass  our  resources  to  attack  critical  nodes,  and  to 
protect  the  Security  of  our  own  resources.  Brungess  (1997)  has  contrasted  the 
approach  to  SEAD  during  Vietnam  and  the  Gulf  War.  In  Vietnam  we  were  in  a 
reactive  mode.  We  were  constantly  one  step  behind  developments  of  the 
Vietnamese  IADS.  However,  in  Desert  Storm,  superior  intelligence,  careful 
analysis,  and  advanced  planning  allowed  us  to  set  the  agenda.  We  understood 
the  Iraqi  IADS  better  than  they  did.  As  a  result,  we  were  able  to  mass  our 
resources  to  attack  critical  nodes  in  the  IADS.  This  enabled  the  allied  forces  to 
gain  air  superiority  very  early  in  the  conflict.  It  is  important  to  realize  that  the 
Iraqi  IADS  system  was  technologically  far  superior  to  what  was  faced  in 
Vietnam.  The  Iraqi  IADS  system  was  one  of  the  most  technologically  advanced 
systems  in  the  world.  Our  success  was  not  a  result  of  vastly  superior 
technologies.  Rather,  the  Offensive  posture  was  achieved  as  a  result  of  a 
superior  understanding  of  how  to  use  the  technologies. 

Surprise  has  always  been  fundamental  to  the  "art  of  war."  In  his  work  by 
that  name.  Sun  Tzu  writes: 

All  warfare  is  based  on  deception.  Hence  when  able  to  attack,  we  must  seem 
unable.  When  using  our  force,  we  must  seem  inactive.  When  we  are  near, 
we  must  make  the  enemy  believe  we  are  far  away.  When  far  away,  we  must 
make  him  believe  we  are  near.  Hold  out  baits  to  entice  the  enemy.  Feign 
disorder,  and  crush  him....  Attack  him  where  he  is  unprepared;  appear 
where  you  are  not  expected.  (1966,  p.  12) 

Advances  in  sensor  and  information  technologies  have  enormous 
implications  for  Surprise.  One  obvious  example  is  the  advances  in  sensor 
technologies  that  have  opened  up  the  electromagnetic  spectrum  to  our  (and  our 
enemies')  vision.  Thus,  the  visual  horizon  and  darkness  are  no  longer  sufficient 
cover  to  hide  our  movements  and  intentions  from  the  enemy.  Further,  the  need 
to  communicate  our  intentions  to  widely  distributed  forces  using  the 
electromagnetic  spectrum  and  information/ communication  networks  makes 
these  intentions  vulnerable  to  interception  by  our  enemy.  A  fundamental 
challenge  of  IW  will  be  to  balance  the  need  to  communicate  simple,  well-  defined 
objectives  to  our  allies  and,  at  the  same  time,  to  shield  those  intentions  from  the 
enemy. 


Despite  the  changes  in  the  information  space  within  which  the  games  of 
deception  will  be  played.  Surprise  will  continue  to  hinge  on  fundamental 
conditions  of  human  perception  (see  Watts,  1996).  Ultimately,  we  are  dealing 
with  human  beliefs  and  expectations.  The  nature  of  the  data  will  change,  but  the 
crux  of  the  problem  will  remain  the  same — to  gain  advantage  by  biasing  the 
beliefs  and  expectations  of  the  enemy.  Surprise  is  fundamentally  an  issue  of 
human  cognition. 
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The  art  of  war  requires  an  ability  to  maneuver  the  enemy  into  positions 
of  disadvantage.  Traditionally,  this  position  has  referred  to  a  physical  place  with 
reference  to  geographical  features.  For  example.  Sun  Tzu  writes  that  "we  are  not 
fit  to  lead  an  army  on  the  march  unless  we  are  familiar  with  the  face  of  the 
country — its  mountains  and  forests,  its  pitfalls  and  precipices,  its  marshes  and 
swamps"  (1966,  p.  41).  In  IW,  the  terrain  will  not  be  defined  exclusively  in 
geographical  terms.  It  is  important  to  consider  where  the  mountains,  precipices, 
and  swamps  are  in  the  information  space.  What  aspects  of  the  information 
terrain  facilitate  movement  and  provide  advantages?  What  aspects  of  the 
information  terrain  retard  movement  and  waste  resources?  Klein's  (1997) 
analysis  of  expertise  and,  in  particular,  of  the  barriers  to  expertise  might  be  a  first 
step  to  characterizing  the  information  terrain.  Klein  describes  six  barriers  to 
expertise:  excessive  data,  pre-processed  data,  excessive  procedures,  performing 
formal  analyses,  passive  data  handling  (limited  ability  for  information  seeking), 
and  interfaces  that  obscure  the  big  picture.  In  many  respects,  these  barriers  are 
natural  features  of  the  information  technology  landscape  that  can  only  be 
avoided  by  careful  planning  and  smart  maneuvering. 

A  key  to  victory  in  any  engagement  is  the  ability  to  Mass  or  concentrate 
application  of  power  at  the  decisive  place  and  time.  For  IW,  the  decisive  place 
and  time  may  have  less  to  do  with  enemy  manpower  or  firepower  and  more  to 
do  with  the  structure  of  information  networks.  Information  deniability  may  be 
decisive.  In  attacking  an  IADS,  eliminating  command  and  control  nodes  or 
information  links  may  do  far  more  damage  than  eliminating  radars  or  SAMs. 
Brungess  (1994)  makes  a  strong  argument  that  SEAD  of  the  future  must  define 
the  critical  nodes  with  respect  to  the  distributed  information  system  within  the 
IADS.  He  writes: 

The  hallmark  of  the  emerging  new  age  seems  to  be  the  growth  of  information — 
its  management  and  distribution — as  a  determinant  of  power.  As  a  direct 
consequence  of  the  age  of  computers,  information  itself  may  be  the  key  to 
power  in  the  twenty-first  century.  The  nation  that  can  control  data,  deny  it  to 
others,  or  willfully  shape  it  will  be  preeminent  in  the  wars  of  the  future.  In 
this  context,  SEAD  criteria  may  be  couched  more  in  terms  ofinformation 
denial  and  shaping  than  in  the  more  traditional  concepts  ofradar  destruction 
andJaTnmillg.-  (p.  91;  emphasis  added) 

The  last  principle  to  consider  is  Security.  Information  is  a  resource  that 
needs  to  be  protected  and  defended.  For  many  people  this  is  the  first  issue  that 
comes  to  mind  relative  to  the  construct  of  IW.  Thus,  a  major  component  of  IW  is 
information  assurance,  which  the  Air  Force  defines  as  "measures  to  enhance  and 
protect  friendly  information  and  functions"  (Department  of  the  Air  Force,  1996, 
p.  6).  Security  was  also  a  primary  consideration  of  Sun  Tzu  who  wrote: 
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The  good  fighters  of  old  first  put  themselves  beyond  the  possibility  of 
defeat,  and  then  waited  for  an  opportunity  of  defeating  the  enemy. 

To  secure  ourselves  against  defeat  lies  in  our  own  hands,  but  the 
opportunity  of  defeating  the  enemy  is  provided  by  the  enemy  himself  Thus 
the  good  fighter  is  able  to  secure  himself  against  defeat,  but  cannot  make 
certain  of  defeating  the  enemy.  (1966,  p.  24) 

Consistent  with  Sun  Tzu's  intuitions.  Security  is  the  foundation  for  all 
the  other  functions.  That  is,  if  we  cannot  ensure  the  integrity  of  our  information 
resources — if  we  can't  trust  the  data  or  the  analyses  (information  processes) — 
then  everything  crumbles.  The  command  and  control  structure  falls  apart  if  you 
cannot  trust  the  command  communications  or  if  the  simple,  well-defined 
Objectives  of  the  commanders  are  corrupted  by  noise  in  the  communication 
channels.  If  you  can't  trust  the  intelligence  data  or  the  projections  of  your 
analyses,  then  you  cannot  adopt  an  Offensive  posture  and  you  are  doomed  to  be 
one  step  behind  an  opponent  with  reliable  information  sources.  The  planning 
horizon  will  be  set  by  the  integrity  of  the  information  on  which  it  is  based. 

Thus,  Figure  4  organizes  the  nine  principles  of  war  in  a  way  that  reflects 
three  fundamental  dimensions  of  IW.  At  die  top  is  the  command  and  control 
function.  Our  ability  to  define  and  communicate  clear  and  feasible  objectives 
will  increasingly  depend  on  our  facility  with  information  management  tools.  At 
the  intermediate  level  are  issues  associated  with  counterinformation.  This 
involves  the  real  time  control  of  information  flow.  The  key  is  to  stay  one  step 
ahead  of  the  enemy — to  dictate  the  time  and  place  of  engagements;  to  maneuver 
the  enemy  into  positions  of  disadvantage;  and  to  focus  our  power  where  its 
impact  will  be  most  decisive.  At  the  lowest  level,  the  issue  is  information 
assurance.  Can  we  protect  our  information  sources  from  the  enemy?  Ensuring 
the  integrity  of  information  sources  is  critical  to  all  aspects  of  IW. 

The  top  down  flow  in  Figure  4  does  not  reflect  importance.  The 
principles  should  be  viewed  as  links  in  a  chain.  Every  link  must  bear  its 
appropriate  load  if  the  chain  is  to  function.  However,  there  is  a  sense  in  which 
the  top  down  flow  helps  to  reveal  the  coupling  across  the  principles.  The  top  of 
the  diagram  tends  to  reflect  strategic  constraints.  There  is  increasing  demand  to 
approach  warfare  in  an  integrated,  as  opposed  to  piecemeal,  fashion.  Brungess 
(1994)  makes  the  argument  in  the  context  of  SEAD: 

The  integrated  approach  represents  the  evolutionary  trend.  Continued 
piecemeal  applications,  no  matter  how  desirable  they  may  be,  must  be  viewed 
in  the  reality  of  the  evolutionary  trend. ...  Objectives  need  to  be  defined  as  a 
function  of  the  integrated  use  of  concepts  and  SEAD  weapons,  not  only 
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because  doing  this  makes  sense,  but  because  it  represents  the  most  appropriate 
adaptation  to  changing  realities,  (p.  82-83) 

War  can  no  longer  be  planned  piecemeal  around  land,  sea,  and  air  (or  around 
particular  component  threats).  The  services  must  function  as  a  coordinated  and 
complementary  whole.  This  is  why  Unity  of  Command  is  at  the  top  of  Figure  4. 
Strategies  must  be  formulated  to  coordinate  across  forces.  These  strategies  will 
have  to  generate  clearly  defined  Objectives  that  are  feasible  (relatively  Simple 
and  realistic/given  the  economic  constraints). 

The  bottom  up  flow  in  Figure  4  emphasizes  the  tactical  constraints.  That 
is,  secure  information  (Security)  will  be  the  basis  for  tactical  decisions  that  will 
allow  us  to  hide  our  intentions  from  the  enemy  (Surprise),  to  determine  the  time 
and  place  for  action  (Maneuver),  and  to  focus  these  actions  where  they  will  have 
the  largest  impact  (Mass).  These  are  the  foundations  for  proactively  (Offensive) 
achieving  the  strategic  goals. 

Viewed  from  the  same  perspective  but  with  somewhat  different 
semantic  labels,  we  must  first  gain  control  of  the  information  realm.  Given  that 
foundation,  we  can  then  proceed  to  exploit  this  "high  ground"  through  a  wide 
variety  of  offensive  and  defensive  information  operations.  These  capabilities  and 
actions  will,  in  turn,  serve  to  enhance  our  actions  across  virtually  the  entire 
spectrum  of  the  conflict. 


Abstract  Function 

The  fundamental  issue  at  the  Abstract  Function  level  is  the  issue  of 
measurement.  In  some  sense,  this  level  addresses  the  "currency  exchange 
problem"  between  the  ends  at  the  Functional  Purpose  level  and  the  means  defined 
at  lower  levels  within  the  abstraction  hierarchy.  The  problem  is  to  formulate  or 
identify  an  abstract  set  of  dimensions  (measurements)  that  will  allow  means  to 
be  scored  or  measured  with  respect  to  the  desirable  goals  or  values  of  the  system. 
Without  a  common  reference,  it  would  be  impossible  to  compare  and  evaluate 
alternative  means.  It  would  be  impossible  to  predict  whether  a  solution  was 
feasible  or  desirable  in  comparison  to  other  candidate  solutions.  The  goal  at  this 
level  of  analysis  is  to  generate  a  low-dimensional  description  that  will  facilitate 
the  classification  of  solutions  with  respect  to  their  feasibility  and  with  respect  to 
their  relative  costs  and  benefits. 

Another  way  to  think  about  the  problem  of  Abstract  Function  in  military 
terms  is  to  consider  the  link  between  strategy  and  doctrine  (discussed  at  the  level 
of  Functional  Purpose)  and  operational  art  and  tactics.  "Strategy  and  doctrine 
are  conceptual;  operational  art  and  tactics  are  pragmatic"  (Brungess,  1994;  p. 

136).  Strategy  and  doctrine  set  the  goals  and  norms  (prescriptions);  tactics 
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represents  the  means.  The  problem  at  the  Abstract  Function  level  is  to  identify 
dimensions  for  measuring  tactics  that  allow  these  tactics  to  be  compared  and 
evaluated  relative  to  strategic  objectives.  If  this  can  be  done,  then  these  measures 
will  be  useful  in  the  design  process.  They  will  provide  a  basis  for  predicting  the 
"best  way." 

How  do  you  evaluate  systems  relative  to  the  goals  for  IW?  Brungess 
(1994)  articulates  the  measurement  problem  in  the  context  of  SEAD: 

SEAD  effectiveness  in  Vietnam  was  measured  by  counting  destroyed 
SAM  sites  and  radars.  Applying  that  criteria  to  the  SEAD  technologies 
used  in  Desert  Storm  yields  a  confused,  possibly  irrelevant  picture. 

SEAD  weapons  and  tactics  evolution  has  outpaced  the  development  of 
criteria  to  measure  SEAD's  total  contribution  to  combat,  (p.  51-52) 

Brungess  (1994)  contrasts  four  approaches  to  measurement — historical, 
engineering,  commonsense,  and  object-based.  With  the  historical  approach  the 
past  becomes  the  measure  of  the  future — that  is,  successes  in  the  past  are 
projected  into  the  future.  You  use  what  worked  in  the  past.  Also,  "lessons 
learned"  from  past  failures  might  inform  design  for  the  future.  In  stable  times,  it 
may  be  safe  to  generalize  from  the  past  to  the  future;  however,  in  a  time  of 
revolutionary  technological  breakthroughs,  the  past  may  not  be  representative  of 
the  future.  The  engineering  approach  focuses  on  quantitative  measures  (e.g., 
probability  of  kills,  probability  of  survival).  However,  in  order  to  get  stable  and 
comparable  numbers,  the  engineering  approach  tends  to  depend  on  a  narrow 
range  of  objective  tests.  These  tests  typically  are  chosen  to  meet  the  constraints 
of  measurement,  and  are  often  based  on  naive  assumptions  about  the  operational 
context.  The  commonsense  approach  to  measurement  is  a  reaction  to  the  failures 
of  the  engineering  approach.  Decision  makers  have  grown  suspicious  of  the 
quantitative  models  used  by  the  engineering  approach.  The  commonsense 
approach  emphasizes  simplicity  and  consensus.  The  opinion  and  recognition  of 
the  decision  makers  become  the  criteria  for  measuring  the  "goodness"  of  a 
design.  The  assumption  is  that  decision  makers  can  recognize  a  good  solution 
when  they  see  it.  Thus,  subjective  pilot  (or,  more  generally,  operator)  opinion 
may  be  the  principal  measure  of  effectiveness. 

The  fourth  approach  to  measurement,  which  Brungess  (1994)  advocates 
as  the  preferred  approach,  is  the  objective  approach.  For  Brungess  the  term 
"objective"  is  synonymous  with  "goal"  or  "function."  This  approach  requires  a 
shift  of  focus  from  the  activity  of  the  system  (e.g.,  number  of  lulls)  to  the 
constraints  of  the  operational  space.  That  is,  the  measures  should  be  defined 
with  respect  to  functional  goals.  Thus,  for  example,  the  criteria  for  evaluating 
systems  for  SEAD  must  reflect  the  operational  goals.  The  first  questions  that 
must  be  asked  are  what  are  the  "objectives"  of  the  SEAD  mission  and  what  are 
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the  constraints  to  achieving  those  objectives.  One  quickly  realizes  that  "'bombs 
on  target'  no  longer  carries  the  certainty  of  desired  system  destruction  that  it 
once  did"  (p.  169).  Today,  the  objective  for  the  SEAD  mission  depends  more  on 
information  deniability  than  on  destroying  SAMs,  and  this  must  be  reflected  in 
measurement. 


For  information  warfare,  then,  the  question  is,  how  do  you  measure 
facility  in  managing  information?  The  rulers  used  must  reflect  both  the 
principles  of  war  and  the  pragmatic  constraints  on  the  activities  that  might  serve 
these  principles.  The  measures  should  help  to  differentiate  an  effective 
information  processor  (e.g.,  expert)  from  a  less  effective  information  processor 
(e.g.,  novice).  The  measures  should  also  provide  insights  as  to  what  attributes  of 
the  systems  make  a  difference.  To  a  large  extent,  systems  for  IW  are  "expert" 
systems.  Thus,  the  Abstract  Function  that  must  be  measured  is  expertise. 

At  an  abstract  level,  the  problem  of  IW  is  fundamentally  a  problem  of 
expertise.  Of  the  many  attributes  that  differentiate  novices  and  experts,  two  that 
might  be  particularly  suited  to  the  measurement  problems  associated  with  IW 
are  the  speed  at  which  they  perform  and  the  robustness  of  their  performance. 
Experts  tend  to  be  much  quicker  to  reach  a  quality  solution.  Research  in 
numerous  fields  of  expertise  show  that  experts  generate  high  quality  alternatives 
very  early  (often  as  the  first  option  considered)  relative  to  novices.  Also,  experts 
tend  to  be  more  adaptive  (robust);'that  is,  they  are  more  flexible  in  the  face  of 
changing  demands.  Figure  5  illustrates  how  measures  of  speed  and  flexibility 
might  help  to  identify  qualitatively  different  information  processing  systems. 

Automatons  are  fast,  but  rigid.  Automatons  achieve  speed  by  taking 
advantage  of  local  consistencies.  However,  these  local  consistencies  are 
bounded.  In  other  words,  automatons  utilize  heuristics  or  short  cuts  that  will 
only  work  in  particular  contexts.  These  systems  break  down  if  the  context 
changes.  Note  that  automatons  will  function  like  experts  within  a  narrow  scope. 
If  context  is  stable,  automatons  are  good  solutions  to  the  information  processing 
problem.  An  autopilot  is  an  example  of  an  automaton.  An  autopilot  can  be 
tuned  to  optimally  fly  an  aircraft,  as  long  as  the  assumptions  about  aircraft 
dynamics  and  the  environment  that  were  designed  into  the  control  algorithms 
hold.  If  the  dynamics  change  (for  example,  due  to  damage  to  a  flight  surface)  or 
the  environment  changes  (for  example,  wind  shear)  in  ways  that  the  designers 
did  not  anticipate,  most  autopilots  will  not  be  able  to  reliably  compensate  to  the 
changes.  One  of  the  problems  with  Taylorism  was  that  the  focus  on  efficiency 
tended  to  drive  systems  toward  performing  like  "well-oiled  machines"  and 
workers  tended  to  be  seen  as  cogs  within  the  machine.  The  result  has  been  an 
efficient  system,  but  a  very  brittle  one  that  cannot  easily  adapt  to  changing 
environments  (e.g.,  changing  markets).  "  ° 
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Figure  5:  A  two-dimensional  space  (decision-cycle  speed  x 
robustness/adaptability)  for  characterizing  the  performance  of 
intelligent  agents. 


Formal  systems  are  extremely  flexible.  That  is,  formal  systems  can  be 
applied  across  a  wide  range  of  situations.  However,  generality  (flexibility)  is 
achieved  at  the  price  of  high  computational  load.  These  systems  are  not  capable 
of  taking  advantage  of  local  constraints  that  might  allow  short  cuts.  Formal 
systems  perform  like  experts  when  time  is  unlimited,  but  these  systems  break 
down  under  time  pressure.  For  example,  a  formal  systems  approach  to  a  chess 
algorithm  would  involve  an  exhaustive  search  of  the  space  of  possible  moves. 
Even  with  great  advances  in  processor  speed,  the  combinatorics  of  chess  set 
pragmatic  time  boundaries  as  to  how  deep  a  search  is  possible  (thus  bounding 
the  intelligence  of  the  program).  Most  advanced  chess  programs  are  not  purely 
formal  systems.  The  programs  incorporate  heuristics  derived  from  observations 
of  human  experts. 

The  choice  of  time  and  robustness  as  key  dimensions  of  IW  is  not  simply 
motivated  by  intuitions  about  expertise,  but  also  reflects  intuitions  about  the 
fundamental  nature  of  war.  Watts  (1996)  chose  two  metrics  for  making  historical 
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comparisons  relative  to  the  question  of  whether  the  "friction"  of  war  is  being 
reduced  as  a  result  of  advances  in  information  processing  technologies.  These 
dimensions  were  "decision-cycle  times"  and  "option  sets  in  possibility  space." 
These  dimensions  seem  quite  consistent  with  the  framework  in  Figure  5.  The 
relation  between  decision-cycle  time  and  speed  is  obvious.  The  concept  of 
"option  sets  in  possibility  space"  may  be  less  obvious,  but  there  is  a  connection. 
The  option  sets  in  possibility  space  represents  the  number  of  feasible  or  possible 
actions  (in  other  words,  the  degrees  of  freedom).  The  number  of  possible  actions 
can  be  bounded  by  the  situation.  For  example,  the  terrain  might  constrain  the 
feasible  directions  for  attack  or  escape.  The  number  of  possible  actions  can  also 
be  bounded  by  internal  constraints.  Bounded  assumptions  incorporated  in 
automatons  are  an  example  of  an  internal  constraint  on  action.  Thus,  the  more 
robust  an  information  processing  system,  the  less  its  degrees  of  freedom 
(options)  are  constrained.  Thus,  Watts'  (1996)  intuitions  about  the  friction  of  war 
seem  to  be  consistent  with  intuitions  about  expertise.  A  system  will  be  more 
expert  in  dealing  with  the  friction  of  war  to  the  extent  that  the  system  can 
implement  decisions  quickly  without  artificially  constraining  the  possibilities. 

If  we  return  to  the  expertise  literature  there  is  an  apparent  contradiction. 
Observations  of  experts  (e.g.,  chess  players)  show  that  experts  consider  many 
fewer  options  before  choosing  an  alternative  than  do  novices.  One  grand  master 
is  reported  to  have  responded  to  the  question  of  how  many  options  he 
considered  before  making  a  move  as  -follows:  "Just  one,  but  it's  the  best  one!" 
This  is  only  an  apparent  contradiction.  This  example  illustrates  the  difference 
between  tacit  and  explicit  knowledge.  Experts  do  not  explicitly  compare  and 
contrast  as  many  alternatives  as  novices.  However,  this  does  not  mean  that  the 
experts  have  fewer  alternatives.  The  fact  that  experts  consistently  generate  the 
"best"  alternative  suggests  that  many  options  are  being  considered.  It  seems  that 
experts  are  able  to  zero-in  on  the  correct  alternative  using  tacit  perceptual  or 
recognitional  processes  (e.g.,  Klein,  1993).  The  case  might  be  made  that  it  is  the 
efficiency  of  the  tacit  perceptual  or  recognitional  processes  that  allows  the  expert 
to  be  both  fast  and  robust  in  processing  information. 

So,  speed  (or  decision-cycle  time)  and  robustness  (or  the  number  of 
options  in  possibility  space)  may  be  two  fundamental  dimensions  that  ought  to 
be  reflected  in  performance  measures.  However,  these  measures  may  be  too 
gross  for  many  design  and  evaluation  questions.  It  might  be  useful  to  consider 
what  attributes  of  information  processing  systems  create  the  possibility  for  speed 
and  robustness.  Here  is  where  Klein's  (1997)  analysis  of  barriers  to  expertise 
might  help  to  link  the  characteristics  of  expertise  to  the  characteristics  of 
information  technologies.  In  many  cases,  these  technologies  are  designed  to 
function  either  as  formal  systems  or  automatons.  However,  developments  in 
graphical  displays  and  visualization  technologies  (e.g.,  virtual  reality  [VR])  may 
provide  an  opportunity  to  enhance  perceptual  and  recognitional  processes  that 
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appear  to  be  fundamental  to  expertise.  In  Whitaker  and  Kuperman's  (1996) 
terms,  expertise  may  depend  on  the  availability  of  a  Common  Battlespace  Picture 
that  shows  the  "full  space  of  possibilities"  and  supports  recognitional  processes 
so  that  the  best  option  within  this  space  can  be  "seen." 

Decision-cycle  time  and  robustness  represent  the  negative  and  positive 
aspects  of  friction.  Reducing  friction  will  generally  result  in  faster  decision-cycle 
times,  but  less  robustness.  Increasing  friction  will  result  in  slower,  but  often 
more  reliable  (robust)  decisions.  The  expert  is  capable  of  faster  decisions,  but  this 
does  not  mean  that  the  expert  is  always  moving  forward  at  the  fastest  pace.  Part 
of  the  skill  is  to  "go  quick,  slowly."  For  example,  novices  in  sports  often  rush 
their  actions  (and  novice  readers  and  speech  makers  often  speak  too  fast).  The 
key  is  not  how  fast  you  go,  the  key  is  to  be  able  to  respond  at  the  right  time. 
Sometimes,  it  is  the  expert's  ability  to  slow  down— to  be  mindful  or  heedful  of 
potential  dangers  or  to  recognize  when  to  wait  for  information  that  is  missing— 
that  contributes  to  increased  reliability.  The  experts  can  go  fast,  but  they  don't 
have  to  go  fast.  They  also  recognize  when  it  is  appropriate  to  put  on  the  brakes 
and  to  wait  for  more  information  (let  the  defense  commit  before  you  take  your 
shot).  This  attribute  of  expertise  is  often  referred  to  as  "metacognition"  (see 
Dominguez,  1997  for  an  interesting  discussion  of  the  role  of  metacognition  in 
surgical  decision  making).  This  heedful  attitude  in  which  decisions  are 
continually  monitored,  reviewed,  and  critiqued  is  a  common  attribute  of  high- 
reliability  systems.  As  Rochlin  (1993)  notes  "bubbles  cannot  be  acquired  quickly 
or  simply,  and  they  require  elaborate  and  overlapping  networks  of 
communications  for  their  establishment  and  maintenance"  (p.  215).  Rochlin 
reports  that  tactical  operations  shifts  on  ships  will  sometimes  overlap  by  up  to  an 
hour  to  ensure  that  the  "bubble  is  transferred  smoothly  to  the  succeeding  tactical 
officer  without  a  potentially  dangerous  break  in  routine  or  perception"  (p.  215). 

At  a  trivial  level,  all  work  domains  are  about  expertise.  For  example, 
nuclear  power  plant  operation  depends  on  the  expertise  of  the  operators  and 
medicine  depends  on  the  expertise  of  doctors  and  nurses.  However,  these  work 
domains  are  not  fundamentally  about  managing  information — they're  about 
managing  power  flow  (mass  and  energy)  and  about  managing  health  (anatomy 
and  physiology).  IW ,  on  the  other  hand,  is  fundamentally  about  managing 
information.  It  is  about  having  a  clearer,  more  accurate  view  of  the  world  than 
your  enemy.  It  is  about  shaping  your  enemy's  perceptions  and  expectations.  In 
Sun  Tzu's  terms,  it  is  about  "defeating  your  enemy's  plans"  (1966).  If  the 
information  war  is  won,  then  the  need  for  fighting,  destruction,  and  bloodshed 
will  be  minimized. 
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General  Function 


The  next  level  of  abstraction  considers  IW  in  terms  of  the  flow  of 
information  and  action.  At  this  level  of  analysis,  systems  are  generally  described 
in  terms  of  networks  of  connected  processing  stages,  where  each  processing 
stage  can  be  described  either  in  quantitative  terms  (a  transfer  function)  or 
qualitative  terms  (process  limitations  and  general  properties  of  inputs  and 
outputs).  As -Whitaker  and  Kuperman  (1996)  have  noted,  probably  the  most 
cited  theoretical  construct  in  the  IW  literature  is  the  OODA  Loop  of  Boyd  (1987). 
The  OODA  Loop  is  a  closed-loop  organization  with  the  stages  Observe,  Orient, 
Decide,  and  Act,  as  shown  in  Figure  6.  The  OODA  Loop  emphasizes  the 
dynamic,  iterative  nature  of  information  processing.  The  iterative  or  circular 
coupling  between  perception  and  action  have  long  been  overlooked  in  basic 
studies  of  cognition.  Flowever,  several  recent  models  show  an  increased 
appreciation  for  the  iterative  nature  of  cognitive  processes.  These  include 
Neisser's  (1976)  perception-action  cycle,  which  is  often  cited  in  the  literature  on 
situation  awareness;  Klein's  (1993)  recognition-primed  decision  (RPD)  model  of 
naturalistic  decision  making  (shown  in  Figure  7);  and  Rasmussen's  (1986) 
decision  ladder  (see  also  Flach  &  Rasmussen,  in  press). 

The  cyclic  nature  of  the  IW  problem  has  also  been  reflected  in  analyses  of 
military  missions.  Figure  8  (adapted  from  Brungess,  1994,  Figure  17)  contrasts 
the  vertical  model  of  information  flow  that  reflects  a  traditional  view  of  war  with 
the  cybernetic  model  that  he  feels  is  a  much  more  accurate  characterization  of 
modem  warfare. 

The  cyclic  nature  of  cognition  is  a  critical  aspect  of  the  dynamics  of 
information  processing  and  has  often  been  de-emphasized  in  the  basic  cognitive 
research  literature  that  has  tended  to  treat  information  processing  as  a  linear 
(horizontal)  sequence  of  independent  stages.  It  has  also  been  de-emphasized  in 
classical  models  of  warfare  that  treat  the  flow  of  information  as  a  linear  (vertical) 
flow  of  commands  and  actions.  However,  it  is  becoming  increasingly  clear  that 
any  model  of  IW  systems  at  the  functional  level  will  have  to  address  the  iterative 
nature  of  the  process.  While  the  OODA  Loop  recognizes  the  circular  coupling,  it 
is  far  too  gross  (too  holistic)  to  be  useful  for  characterizing  the  flow  of 
information  and  action  in  a  way  that  can  inform  design  decisions. 
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Figure  6:  The  OODA  Loop:  Observe,  Orient,  Decide,  Act. 


RPD  Model 


Figure  7:  Klein's  RPD  model  of  naturalistic  decision  making 
illustrates  the  closed-loop  coupling  of  perception  and  action: 
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Figure  8:  The  military  command  structure  as  a  rigid,  vertical 
hierarchy  or  as  a  dynamic  system  of  interacting  elements  (Adapted 
from  Brungess,  1994). 

A  more  detailed  look  at  the  information  flow  shows  many  coupled  OODA 
Loops  operating  at  different  time  constants  and  reflecting  different  levels  of 
analysis  (e.g.,  Whitaker  &  Kuperman's  1996  analysis  of  Scud  hunting).  In  his 
classic  text  on  Electromagnetic  Conflict,  Fitts  (1980)  distinguishes  three  different 
analysis  loops:  ELINT  (electronics  intelligence),  ESM  (electronic  support 
measures),  and  ECM  (electronic  countermeasures;  Figure  9).  [Note  that  the  terms 
ELINT,  ESM,  and  ECM  have  evolved  since  Fitts'  (1980)  characterization  of  the 
systems.  The  nested  loops  are  presented  here  to  give  a  sense  of  what  the  nesting 
of  OODA  loops  might  look  like.  This  is  not  intended  to  be  a  definitive  model  of 
electromagnetic  warfare.]  The  ELINT  loop  represents  information  that  is 
collected  over  long  time  periods  (weeks  to  years)  and  that  is  subject  to  extensive 
analysis  and  evaluation.  This  loop  is  involved  in  long  range  strategic  planning. 

It  involves  a  long-term  analysis  of  allied  and  enemy  capabilities  ahd  tendencies 
that  helps  to  determine  long  range  objectives  and  general  strategies. 
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Figure  9:  Electromagnetic  warfare  can  be  viewed  as  a  nested  set  of 
control  loops.  The  outer  loops  have  a  longer  time  constants. 


The  ESM  loop  operates  with  a  smaller  time  constant  (hours  to  weeks). 

The  ESM  loop  reflects  the  information  that  a  tactical  commander  would  gather  in 
order  to  prepare  for  tomorrow's  mission.  It  might  include  input  from  regularly 
scheduled  reconnaissance  missions  or  from  debriefs  of  the  previous  mission.  The 
ESM  loop  represents  tactical  planning.  It  involves  integration  of  highly 
processed  data  available  from  ELINT  with  raw  data  (with  perhaps  some  low 
level  analyses)  obtained  from  recent  observations  of  the  situation.  Some  of  the 
data  from  the  current  observations  are  also  passed  up  to  be  integrated  with  other 
information  as  part  of  the  ELINT  loop. 

Finally,  there  is  the  ECM  loop.  This  loop  has  a  much  smaller  time 
constant  (milliseconds  to  hours).  This  loop  involves  the  moment-to-moment 
actions  of  a  pilot,  a  field  commander,  or  an  individual  soldier  in  response  to 
direct  observations  of  the  enemy.  Performance  within  this  loop  reflects  the  plans 
and  preparations  generated  in  the  ESM  loop  and  the  data  directly  available  to 
the  warfighter. 

It  is  the  couplings  and  time  constants  of  these  interacting  loops  that  begins 
to  provide  a  context  for  answering  questions  about  the  "right  information"  and 
the  "right  time."  The  general  OODA  Loop  is  far  too  gross  a  formulation  for 
addressing  these  questions.  A  critical  question  is  how  can  these  loops  (each 
operating  at  a  different  time  constant  and  on  different  kinds  of  data)  interact  to 
build  a  unified  Common  Battlespace  Picture  that  will  support  full*  consideration 
of  options  and  timely  decision  making  at  all  levels? 
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Figure  10  (adapted  from  Homer,  1996)  shows  some  of  the  nested  loops 
involved  in  IW.  Note  the  "push"  and  "pull"  of  information  in  this  diagram.  This 
emphasizes  that  the  Common  Battlespace  Picture  is  a  joint  function  of  the 
available  data  and  of  the  questions  that  analysts  choose  to  ask.  This  is  a  critical 
aspect  of  the  dynamic.  All  the  data  may  be  available  to  construct  an  accurate 
visualization  of  the  battlespace;  however,  an  accurate  visualization  may  not  be 
realized  because  the  right  questions  were  not  asked.  In  CSE,  the  terms  "data" 
and  "information"  are  sometimes  contrasted  to  make  this  point.  Data  are 
whatever  is  available  from  the  sensors,  but  "information"  is  the  answer  to 
questions  the  analysts  are  asking.  Thus,  information  can  be  missing,  due  either 
to  a  lack  of  data  or  to  a  failure  to  ask  the  right  question. 

An  implicit  assumption  of  the  loops  in  Figures  6-10  is  that  there  is  a 
problem  to  be  solved.  The  emphasis  is  on  problem  solving!  However,  for  IW, 
particularly  in  the  post  cold  war  world  era  where  the  threats  are  ill-defined,  the 
main  problem  may  not  be  problem  solving,  but  problem  finding.  Figure  11, 
adapted  from  Klein  Associates  (1997),  explicitly  includes  problem  detection 
within  the  representation  of  the  perception-action  loop.  Problem  finding  is 
implicitly  recognized  in  Figure  10  as  an  important  aspect  of  "information  pull." 
As  the  following  quote  from  Klein  Associates  (1997)  illustrates,  for  systems  to 
function  skillfully  the  analysts  must  be  actively  engaging  the  data  (information 
pull): 

[an]  important  factor  is  whether  the  problem  solver  forms  his/her  own 
understanding  or  just  accepts  the  description  provided  by  someone  else  (or  by 
a  decision  support  system ).  Having  an  active  engagement  in  building  up 
one's  own  interpretation  is  important  for  detecting  early  signs  of  a  problem. 

Failures  to  detect  problems  result  from  lack  of  experience,  but  they  also  occur 
when  people  explain  away  early  signs  of  the  problem,  and  when  they  show  an 
"automation  bias"  by  passively  accepting  the  outputs  of  a  computer-based 
system,  (p.  2) 
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Figure  10:  A  hypothetical  view  of  the  general  functional  flow  for  IW 
(adapted  from  Homer,  1996). 


Figure  11:  Explicit  representation  of  problem  finding  as  part  of  the 
closed-loop  perception-action  coupling  (from  Klein  Associates,  1997). 
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Physical  Function 


At  this  level  of  analysis,  consideration  is  given  to  allocation  of  the 
various  functions.  A  fundamental  issue  at  this  level  concerns  what  tasks  or 
functions  to  allocate  to  humans  and  what  tasks  to  allocate  to  computers  and 
automatic  control  systems.  Table  2  (from  Sheridan,  1996)  shows  some  of  the 
logical  ways  that  machines  and  humans  might  be  coupled.  Certainly,  IW  is  well 
beyond  Level  1  of  Sheridan's  scale.  The  question  is,  how  much  beyond  this  level 
can  we  venture?  How  much  can  we  rely  on  information  technologies  and  how 
much  authority  can  we  give  to  our  computational  engines?  Appropriate 
decisions  about  automation  can  enhance  expertise,  but  inappropriate  allocations 
may  result  in  barriers  to  expertise.  This  will  not  be  a  simple  question  that  can  be 
answered  once  and  for  all.  Pronouncements  such  as  "a  human  will  always  have 
the  final  authority"  are  naive.  The  question  of  the  degree  of  automation  must  be 
addressed  over  and  over  again  within  each  of  the  many  nested  OODA  loops  that 
make  up  the  dynamic  system  of  IW  and  must  be  reassessed  in  the  context  of 
continuing  advances  in  technology. 


Table  2:  Scales  of  degrees  of  automation 

_ Scale  of  Degrees  of  Automation  (from  Sheridan  1996) _ 

1.  The  computer  offers  no  assistance;  the  human  must  do  it  all. _ 

2.  The  computer  offers  a  complete  set  of  action  alternatives,  and _ 

3.  Narrows  the  selection  down  to  a  few,  or _ 

4.  Suggests  one,  and _ .  _ 

5.  Executes  the  suggestion  if  the  human  approves,  or _ 

6.  Allows  the  human  a  restricted  time  to  veto  before  automatic  execution,  or 

7.  Executes  automatically,  then  necessarily  informs  the  human,  or _ 

8.  Informs  him  or  her  after  execution  only  if  he  or  she  asks,  or _ 

9.  Informs  him  or  her  after  execution  if  it,  'the  computer,  decides  to. _ 

10.  The  computer  decides  everything  and  acts  autonomously,  ignoring  the 
I  human. 


Table  3  (from  Flach,  Eggleston,  Kuperman,  &  Dominguez,  1998)  outlines 
some  important  considerations  with  respect  to  the  function  allocation  problem. 
The  question  is,  which  combinations  of  systems  will  result  in  the  most  efficient 
coordination  across  the  various  control  loops?  What  combination  of  physical 
components  will  result  in  the  greatest  expertise,  permit  the  shortest  decision- 
cycle  times  and  the  greatest  flexibility  or  adaptability?  What  allocations  will 
minimize  the  barriers  to  expertise? 
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Table  3:  Key  considerations  for  function  allocation 


Level 

Key  Considerations  for  Function  Allocation 

Functions 

Priority.  Relative  importance  or  criticality  of  a  particular  function.  Is  it 
essential  or  discretionary  with  respect  to  the  overall  mission  objectives? 
Precedence  Constraints.  Sequential  dependencies  across  the  functions. 

To  what  extent  do  functions  depend  on  the  successful  prior  completion 
of  other  functions? 

Temporal  Constraints.  Are  there  fixed  windows  of  opportunities  or 
fixed  time  schedules  associated  with  particular  functions? 

Spatial  Constraints.  Does  performance  of  the  function  depend  on 
being  at  a  particular  place  or  location? 

Agents 

Structural  Understanding.  Does  the  agent  have  the  appropriate 
knowledge  or  know  how  (e.g.,  right  training  for  operators  or  right 
programming  for  automation)  to  accomplish  the  functions? 

Information  Availability.  Does  the  agent  have  adequate  information 
(good  situation  awareness)  for  completing  the  function? 

Resource  Availability.  Does  the  agent  have  adequate  information 
processing  capacity  (e.g.,  attention,  memory,  computational  power)  or 
physical  capacity  (e.g.,  free  hand)  to  accomplish  the  function? 

Effector  Capability.  Is  the  agent  capable  of  performing  the  actions 
necessary  to  complete  the  function?  Can  the  agent  respond  in  time  with 
the  appropriate  precision? 

Team 

Shared  Mental  Models.  Do  the  cooperating  agents  have  a  common  or 
compatible  vision  (e.g.,  common  assumptions,  shared  rationality)  of  the 
problem  domain? 

Information  Coupling.  Is  there  sufficient  information  for  each  agent  to 
know  what  and  how  other  agents  are  doing? 

Effector  and  Resource  Complementarity.  Do  the  effector  and  resource 
capabilities  of  the  team  span  the  demands  of  the  work  domain? 

Authority  Structure.  How  are  dynamic  decisions  about  allocation 
handled  (e.g.,  central  authority,  first-come,  consensus,  volunteer,  etc.)? 

How  are  conflicts  resolved? 

A  reasonable  hypothesis,  in  the  context  of  our  understanding  of 
expertise,  is  that  efforts  should  focus  on  bringing  the  data  closer  to  critical 
decision  points.  Two  very  different  solutions  are  currently  being  explored.  One 
solution  is  to  bring  the  data  forward,  so  that  it  is  more  readily  available  to  the 
warfighter.  The  Airborne  Warning  and  Control  System  (AWACS)  and  the  Joint 
Surveillance  Target  Attack  Radar  System  (JSTARS)  are  two  examples  where 
information  is  moved  forward  so  that  warfighters  have  more  direct  access  to 
ELINT  and  ESM.  Another  solution  is  to  move  the  decision  points  back  to  the 
information  sources.  Uninhabited  combat  aerial  vehicles  (UCAVs)  are  an 
example  of  this  approach.  That  is,  the  warfighter's  role  (pilot)  is  automated  and 
the  decisions  are  moved  back  to  a  command  center  where  ELINT  and  ESM  data 
are  more  easily  accessed  and  displayed. 
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Which  will  be  the  best  solution — moving  the  data  forward  or  moving  the 
control  back?  A  major  issue  to  consider,  relative  to  moving  the  data  forward,  is 
the  issue  of  cognitive  workload.  There  is  a  danger  that  the  warfighter  will  be 
overwhelmed  with  data.  A  key  consideration  will  be  the  form  or  the 
representation.  If  representations  can  be  designed  that  support  recognitional 
processes,  then  more  data  will  not  necessarily  mean  more  workload.  A  major 
concern,  relative  to  moving  control  back,  is  that  the  automation  of  the  ECM  role 
will  reduce  the  flexibility  in  that  loop.  Key  considerations  will  be  the  time 
constraints  on  communications  from  the  control  center  to  the  UCAV  and  the 
sophistication  of  the  automatic  control  algorithms  that  can  be  implemented  in  the 
UCAV.  Ultimately,  this  will  not  be  a  simple  dichotomous  choice — it  will  be  a 
question  of  when /where /how  can  data  be  moved  forward  and  how  far  forward; 
and  when/where/how  can  control  be  moved  back  and  how  far  back.  Again,  the 
issue  is  to  get  the  right  data  to  the  right  people  at  the  right  time! 

Physical  Form 

At  this  level  of  analysis  the  specific  physical  form  of  the  system  and  the 
specific  trajectories  of  activities  are  considered.  At  the  previous  levels,  questions 
were  framed  in  terms  of  different  aspects  of  the  functional  relations;  in  this 
section  the  pragmatics  of  actually  building  the  physical  system  and  of  physically 
performing  the  activities  are  considered. 

At  a  gross  level  it  is  clear  that  IW  will  not  be  bounded  by  the  same  space- 
time  constraints  that  bounded  conflict  in  the  days  of  Sim  Tzu  or  even  as  recently 
as  World  War  II.  Vietnam  was  fundamentally  different  than  WWII  and  Desert 
Storm  was  different  than  Vietnam.  The  classical  partitioning  of  forces  according 
to  geographical  distinctions  of  land,  sea,  and  air  will  no  longer  be  feasible.  IW 
will  require  an  integrated  joint  forces  approach  to  war.  IW  is  going  to  involve 
managing  widely  (perhaps  globally)  distributed  forces  and  is  going  to  require  a 
degree  of  coordination  and  synchronization  among  those  forces  that  has  never 
been  demanded  in  the  past.  We  can  also  say  with  confidence  that  the  human  will 
remain  a  critical  element  in  IW.  Computers  will  be  essential.  The  full 
electromagnetic  spectrum  will  be  utilized.  Further,  satellites  and  global 
positioning  systems  (GPS)  will  be  involved.  Also,  it  is  clear  that  the  flow  of 
information  will  not  be  bounded  by  military  systems.  International  news 
services  and  global  communication  systems  will  also  be  coupled  more  intimately 
to  the  war  than  ever  before.  Thus,  the  link  between  politics,  public  opinion,  and 
military  action  will  shape  the  dynamic  of  IW.  Already,  we  see  evidence  in  Desert 
Storm  that  managing  mass  media  will  be  part  of  the  IW  problem. 

Already,  it  should  be  apparent  why  this  is  the  base  of  the  abstraction 
hierarchy.  The  number  of  issues  to  consider  at  this  level  of  analysis  is  staggering. 
These  issues  span  many  different  disciplines.  How  is  it  possible  to  deal  with  all 
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the  potentially  relevant  data  at  this  level?  This  is  why  the  abstraction  hierarchy 
is  essential.  The  analyses  at  the  higher  levels  of  abstraction  can  be  used  to  help 
parse  the  problem  at  lower  levels.  Thus,  the  categories  defined  at  higher  levels 
of  abstraction  can  be  used  to  organize  and  integrate  data  at  the  physical  level. 
Having  the  categories  does  not  reduce  the  complexity  of  the  problem,  but  these 
categories  can  facilitate  the  management  of  the  complexity.  The  categories  can 
help  to  guide  and  bound  generalizations.  The  categories  formulated  at  higher 
levels  of  abstraction  provide  a  representational  framework  that  may  facilitate 
expertise  in  making  decisions  about  the  physical  form  of  IW  systems.  There  are 
no  guarantees,  no  simple,  easy  to  understand  recipes  that  will  tell  us  what 
physical  form  IW  systems  should  take.  The  hope,  however,  is  that  by  working 
up  and  down  the  abstraction  hierarchy  we  may  be  in  a  better  position  to  ask  the 
right  questions  and  that  these  questions  will  facilitate  the  iterative  process  of 
generating  and  testing  hypotheses  about  the  physical  form  of  the  system. 

Summary 

Figure  12  uses  the  CSE  Abstraction /Decomposition  space  to  summarize 
the  discussion  in  this  section.  As  noted  earlier,  the  diagonal  within  this  space 
seems  to  provide  the  level  of  description  that  is  most  relevant  to  the  design  of 
systems  that  function  as  experts.  At  the  top  level,  a  general  understanding  of  the 
goals  provides  the  context  for  more  detailed  analyses  at  lower  levels.  At  the 
bottom  level,  every  element  within  the  physical  system  must  be  considered  in 
detail  with  regard  to  its  interface  to  other  pieces  of  the  system.  On  one  hand,  a 
design  based  on  a  good  understanding  of  general  principles  might  fail  because  a 
small  physical  detail  is  overlooked  (for  example,  an  electronic  suite  is  developed 
with  an  interface  that  is  not  compatible  with  other  electronic  systems  in  the 
cockpit).  On  the  other  hand,  all  the  pieces  can  fit  together  well  and  function 
exactly  as  the  designer  intended,  but  if  they  were  not  designed  in  the  context  of 
the  general  functional  requirements  of  the  warfighter,  the  system  may  never  be 
used.  The  designer  solved  the  wrong  problem. 

Figure  12  is  a  map  of  where  we  are  and  the  directions  we  need  to  go  with 
respect  to  a  CSE  analysis  of  IW.  At  the  Functional  Purpose  level  there  is  an 
abundance  of  information  about  the  goals  and  basic  doctrine  of  warfare.  For  the 
most  part,  detailed  analyses  are  not  required  here.  It  is  simply  a  matter  of 
communicating  the  information  in  a  way  that  designers  will  appreciate. 

The  Abstract  Function  level  provides  a  critical  challenge  for  the  analysis 
of  IW.  How  do  we  express  the  functional  requirements  in  terms  that  reflect 
measurable  attributes  of  design  solutions?  Dimensions  like  decision-cycle  time 
and  robustness  provide  a  first-cut,  general  hypothesis  about  dimensions  that 
might  be  useful.  While  these  dimensions  might  be  useful  for  post  hoc 
comparisons  of  systemic  design  solutions,  these  measures  are  probably  too 


43 


general  to  provide  much  guidance  to  designers  of  information  technologies. 
Klein's  (1997)  discussion  of  the  barriers  to  expertise  associated  with  information 
technologies  might  provide  useful  hints  that  might  provide  the  basis  for  a 
higher-dimensional  description  of  the  design  space.  On  the  other  hand,  classical 
engineering  measures  of  probability  of  kill  or  probability  of  survival  are 
probably  too  elemental — in  that  their  measurement  will  typically  require 
restrictive  assumptions  about  the  operational  context. 

The  General  Function  level  also  is  a  challenge.  It  is  clear  that  IW  will 
involve  a  closed-loop  perception-action  dynamic  as  typified  by  the  OODA  Loop. 
However,  it  will  be  a  complex  network  of  interconnected  loops  operating  with 
different  time  constants  and  operating  at  different  levels  of  data  decomposition 
and  integration.  In  order  to  fully  characterize  this  network  it  will  be  necessary  to 
develop  detailed  models  of  prototypical  IW  systems  (e.g.,  Whitaker  and 
Kuperman's  (1996)  analysis  of  Scud  hunting)  and  then,  through  comparisons  and 
contrasts  among  these  systems,  it  may  be  possible  to  abstract  a  picture  of  the 
general  functions  that  govern  the  flow  of  information  and  action  for  IW.  The 
functional  flow  representation  must  make  both  the  "push"  and  "pull"  (e.g., 
problem  finding)  aspects  of  IW  explicit. 

The  Physical  Function  and  Physical  Form  levels  represent  the  pragmatics 
of  the  design  problem.  At  this  point,  analyses  should  begin  with  evaluations  of 
general  solutions  (across  different  domains,  as  typified  by  Sheridan's  list)  and 
particular  solutions  that  exist  for  IW  (e.g.,  JSTARS  and  UAVs).  What  works? 
Where  are  there  difficulties?  These  kinds  of  observations  will  be  useful  for 
understanding  how  the  different  functional  components  (humans  and  machines) 
can  most  effectively  be  coupled  and  will  draw  attention  to  key  elements  of 
physical  form  that  may  be  critical  to  performance. 

It  is  important  to  appreciate  the  fact  that  the  diagonal  of  the  CSE  space  is 
a  target  that  can  only  be  approached  asymptotically.  That  is,  there  will  always  be 
unanswered  questions.  Our  understanding  will  never  be  complete.  Partly,  this 
is  due  to  the  fact  that  technologies  are  not  stationary — they  are  continuously 
evolving.  But,  more  fundamentally,  it  is  due  to  the  nature  of  the  design  and 
discovery  process.  The  CSE  analysis  cannot  be  isolated  as  a  precursor  to  design. 
The  CSE  analysis  must  be  integrated  within  the  iterative  process  of  design.  The 
CSE  analysis  is  a  mechanism  for  generating  hypotheses.  Each  design  is  a  test  of 
our  hypotheses.  The  results  from  each  test  must  be  fed  back  into  the  analysis 
process  so  that  we  can  generate  better  hypotheses.  In  an  important  sense,  CSE  is 
not  an  external  framework  for  evaluating  IW.  Rather,  CSE  is  intrinsic  to  IW. 

CSE  is  the  basis  for  a  proactive  (as  opposed  to  reactive)  approach  to  IW.  CSE  is 
an  integral  framework  for  planning  the  future  of  IW. 
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Figure  12:  Fleshing  out  the  CSE  framework  with  some  observations 
about  IW. 
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V.  FUTURE  DIRECTIONS. 


In  the  previous  section  we  attempted  to  present  an  overview  showing 
where  we  are  and  the  directions  in  which  we  ought  to  be  moving  in  order  to 
design  for  IW  in  the  future.  In  this  final  section,  we  will  present  some 
suggestions  for  how  to  proceed. 

Hypothesis  Testing  vs.  Hypothesis  Generation 

Our  society  places  great  faith  in  the  scientific  method — and  for  many  the 
scientific  method  is  synonymous  with  empirical  experiments.  Therefore,  there  is 
a  common  expectation  that  the  first  step  to  understanding  IW  will  be  to  do  some 
experiments.  The  first  impulse  is  to  generate  data!  So,  if  there  are  questions 
about  a  particular  technology,  then  let's  do  an  experiment  in  which  we  compare 
performance  of  the  new  technology  with  a  control  condition  using  the  old 
technology.  We  test  two  groups  in  an  identical  battle  scenario.  If  the  group  who 
has  the  new  technology  has  a  greater  percentage  of  hits  and/or  has  fewer  losses, 
then  we  can  infer  that  the  new  technology  is  beneficial.  Obviously,  this  is  a  clear, 
straightforward,  wrong  approach  to  the  problem. 

The  first  problem  with  this  approach  is  the  attribution  problem.  Was  the 
difference  (or  lack  of  difference)  in  performance  due  to  differences  in  the 
technology?  Anyone  familiar  with  experimental  design  realizes  that  there  are 
many  potential  confounds  (e.g.,  individual  differences)  that  must  be  dealt  with. 
There  are  ways  to  reduce  the  confounding,  but  even  in  the  simplest  experiments 
the  potential  for  confounding  cannot  be  totally  eliminated.  However,  for 
complex  systems  and  new  technologies  the  attribution  problem  is  further 
complicated  by  the  fact  that  there  is  never  a  simple  dichotomy  (e.g.,  old 
technology  vs.  new  technology).  For  example,  you  can't  do  a  single  experiment 
to  compare  graphical  displays  to  numeric  displays  of  information.  There  are  an 
infinite  number  of  ways  that  a  graphical  display  might  be  implemented  in  a 
design.  So,  it  would  be  a  mistake  to  throw  out  graphical  display  technology, 
because  one  particular  implementation  of  a  graphical  display  did  not  work.  The 
idea  of  a  "critical  experiment"  is  probably  a  fiction  that  we  have  created  in  our 
narrative  retelling  of  the  story  of  science.  It  is  definitely  a  fiction  with  regard  to 
the  problem  of  evaluating  complex  technologies  for  application  in  complex  work 
domains.  To  quote  Alan  Newell  (1973) — "You  can't  play  20  questions  with 
nature  and  win!" 

A  second  problem  with  the  hypothesis  testing  approach  is  the 
generalization  problem.  Is  the  scenario  that  you  choose  for  your  test 
representative  of  the  actual  operational  environment  or  of  the  future 
environments  where  the  system  will  eventually  be  implemented?  Are  the 
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subjects  in  your  experiment  representative  of  the  ultimate  user  population?  If 
the  experimental  context  is  not  representative  of  the  operational  context,  then  the 
differences  observed  in  the  experiment  may  be  totally  irrelevant  to  the  design 
questions  that  originally  motivated  them.  You  may  have  exactly  the  right 
representation  for  the  "toy"  world  employed  in  the  laboratory,  but  it  may  be 
exactly  the  wrong  representation  for  the  operational  problem. 

A  third  problem  with  the  hypothesis  testing  approach  is  the 
measurement  problem.  Are  the  performance  measures  that  you  collect  in  the 
experiment  sensitive  to  the  differences  that  make  a  difference  in  the  operational 
environment?  As  Brungess  (1996)  notes  "SEAD  effectiveness  in  Vietnam  was 
measured  generally  by  counting  destroyed  SAM  sites  and  radars.  Applying  that 
criteria  to  the  SEAD  technologies  used  in  Desert  Storm  yields  a  confused, 
possibly  irrelevant  picture"  (pp.  51-52). 

The  point  to  be  made  here  is  that  a  hypothesis  testing  program  of 
research  may  generate  lots  of  data,  but  it  does  not  allow  you  to  avoid  the 
problems  posed  by  the  Abstraction  Hierarchy.  Without  the  kind  of  conceptual 
framework  provided  by  the  Abstraction  Hierarchy,  the  data  from  a  "twenty 
questions  approach"  will  not  yield  information  relevant  to  the  design  questions. 
Hypothesis  testing  is  a  reactive  mode  of  science.  It  will  generally  lag  well  behind 
the  design  questions  and  will  never  keep  up  with  the  evolution  of  new 
technologies.  For  example,  the  graphical  desktop  interface  on  personal 
computers  was  not  discovered  by  Human  Computer  Interface  (HQ)  researchers. 
It  was  a  product  of  the  design  process.  Laboratory-based  research  in  HCI  is  still 
struggling  to  explain  why  this  interface  works  as  well  as  it  does. 

The  problems  of  design  require  a  proactive  approach  to  science.  The 
emphasis  needs  to  be  on  exploration  of  the  phenomenon,  discovery,  and 
hypothesis  generation.  The  first  step  in  this  program  of  research  is  for 
researchers  to  immerse  themselves  in  specific  phenomena.  The  model  for  this 
kind  of  science  is  anthropology  or  ethology  (not  experimental  physics).  The 
challenge  is  to  enter  a  work  environment  and  to  discover  the  intrinsic  semantics 
of  that  environment.  The  challenge  in  doing  this  is  to  release  our  natural  biases 
and  expectations  about  what  is  logical  so  that  we  can  discover  the  natural 
intrinsic  logic  (eco-logic)  of  the  work  domain.  Again,  remember  the  examples 
from  Rasmussen  and  Klein— expectations  based  on  normative  logics  are 
generally  not  validated  when  tested  against  performance  of  experts  in  their 
natural  domain. 


First  Step 

The  first  step  in  this  program  should  be  to  identify  some  specific 
examples  where  new  technologies  are  being  implemented  in  ways  that  might  be 
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representative  of  the  general  problems  of  IW  (e.g..  Scud  hunting,  AW  ACS, 
UCAVs).  These  specific  applications  then  need  to  be  placed  under  a  microscope. 
That  is,  the  specific  examples  need  to  be  studied  intensively  using  table-top 
analysis  methods,  naturalistic  observation,  and  knowledge  elicitation  techniques 
(see  Flach,  Eggleston,  Kuperman,  &  Dominguez,  1998  for  descriptions  of  these 
methods  in  the  context  of  UCAVs;  see  also  Carnegie  Group,  1997  for  a  review  of 
knowledge  elicitation  tools;  or  Klein,  1997  for  a  discussion  of  naturalistic 
observations  of  teams). 

The  data  acquired  from  these  explorations  must  then  be  integrated  using 
the  CSE  framework.  The  goal  is  to  build  a  CSE  description  of  each  specific 
example.  In  other  words,  the  preliminary  goal  is  to  fill  in  as  much  of  the 
diagonal  of  the  Abstraction /Decomposition  space  as  possible.  This  will  be  an 
iterative  process  of  observing,  analyzing,  and  refining  observations.  Analyses 
must  proceed  simultaneously  at  all  five  levels  of  abstraction.  Further,  it  must 
proceed  in  such  a  way  that  discoveries  at  each  level  inform  (and  are  tested 
against)  discoveries  at  other  levels.  The  initial  tests  of  these  observations  will  be 
the  ability  to  predict  events  in  the  natural  environment  and  the  internal 
consistency  across  levels  of  abstraction. 

As  an  internally  consistent  description  emerges,  then  we  may  be  in  a 
position  to  formulate  reasonable  experimental  tests  of  some  of  our  hypotheses. 
The  most  important  tests  of  our  hypotheses,  however,  will  not  necessarily  be 
laboratory  experiments,  but  rather  they  will  be  designs  (Kirlik,  1995).  The 
ultimate  test  of  the  CSE  analysis  is  whether  the  data  can  be  integrated,  not  as  an 
abstract  data  base,  but  as  concrete  solutions.  The  CSE  paradigm  must  ultimately 
be  tested  against  the  problem  of  design.  To  be  more  precise,  the  value  of  CSE 
will  be  measured  against  our  ability  to  engineer  effective  problem 
representations.  These  representations  will  be  implemented  either  as  interfaces 
(external  representations)  or  as  training  protocols  (internal  representations). 

The  term  interface  is  used  here  in  its  broadest  sense.  This  includes  all  of 
the  technologies  that  stand  between  the  human  operators  and  the  problem  space 
with  which  they  are  interacting.  For  example,  it  includes  displays  and  controls 
(in  all  modalities)  and  any  computational  engines  (e.g.,  decision  aids,  filters, 
simulations,  etc.)  that  might  be  used  in  the  analysis  and  decision  making  process. 

The  term  "training"  includes  classical  forms  of  instruction.  However, 
there  is  clear  evidence  that  traditional  classroom  or  textbook  approaches  to 
instruction  are  not  sufficient  to  developing  expertise.  The  development  of  the 
recognitional  or  perceptual  skills  of  expertise  requires  practice,  practice,  practice! 
Thus,  an  important  part  of  the  CSE  design  effort  will  focus  on  incorporating  the 
developing  understanding  of  the  problem  into  high  fidelity  simulations  where 
operators  can  "exercise"  their  skills  in  conditions  that  are  as  close  to  "real  game" 
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conditions  as  possible.  A  bonus  of  this  process  is  that  the  high-fidelity  training 
simulations  also  offer  a  test-bed  for  empirically  evaluating  interfaces  (and, 
conversely,  high-fidelity  concept  demonstration  facilities,  if  developed  early  in 
an  evolutionary  design  process,  may  be  migrated  into  the  training  support 
system  world  of  application).  These  high-fidelity  simulations  offer  a  middle 
ground  between  the  "toy  worlds"  of  the  bench  scientists  and  the  operational 
world  of  the  field  researcher.  Thus,  they  offer  the  possibility  for  research 
programs  that  can  have  both  high  internal  validity  (control  over  potential 
confounds,  precise  manipulation  of  independent  variables,  and  precise 
measurement)  and  high  external  validity  (the  context  is  representative  of  the 
natural  ecology). 
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VI.  CONCLUSIONS 


War  is  a  complex  and  chaotic  human  endeavor.'  Human  frailty  and 
irrationality  shape  war's  nature.  Uncertainty  and  unpredictability— what 
many  call  the  "fog"  of  war — combine  with  danger,  physical  stress,  and  human 
frailty  to  produce  "friction,"  a  phenomenon  that  makes  apparently  simple 
operations  unexpectedly,  and  sometimes  insurmountably,  difficult. 

Uncertainty,  unpredictability,  and  unreliability  are  always  present,  but  sound 
doctrine,  leadership,  organization,  core  personal  values,  technologies,  and 
training  can  lessen  their  effects.  ( Air  Force  1-1, 1997) 

Will  advances  in  information  technologies  provide  new  tools  for 
managing  the  fog  and  friction  of  war,  or  will  they  further  complicate  the 
problem — exaggerating  uncertainty  and  reducing  reliability?  Certainly,  support 
for  both  sides  of  this  question  is  available.  There  are  many  examples  where 
advances  in  computational  power  and  in  graphical  displays  have  facilitated 
problem  solving  and  led  to  increased  efficiency  and  reliability.  On  the  other 
hand,  one  does  not  have  to  look  far  to  find  examples  of  "clumsy  automation," 
where  technology  is  a  barrier  to  expertise. 

The  path  to  "clumsy  automation"  is  a  seductive  one,  and  there  is  some 
evidence  that  we  may  already  be  heading  down  that  path  (e.g.,  Owens,  1995).  It 
is  important  that  we  heed  this  warning  from  Rochlin  (1993): 

the  attitudes  of  many  of  those  responsible  for  the  design  and  implementation 
of  the  new  systems  have  decidedly  Taylorist  resonances — right  down  to  the 
scientific  belief  that  there  is  indeed  'one  best  way’  to  perform  any  task, 
including  a  managerial  one.  Even  more  striking  is  the  return  to  the  search  for 
central  control  (Beninger,  1986).  In  her  interviewing,  Zuboff(1984)  noted 
several  instances  where  the  manager's  view  of  the  intended  result  was  a 
centralization  of  control  in  order  to  reduce  uncertainty  and  allow  syncretic 
planning  of  all  operations.  As  she  reported,  a  survey  of  plant  managers 
conducted  by  the  Honeywell  Corporation  found  that  "almost  without 
exception,  the  "technology  ideal"  reported  by  plant  managers  was  having  one 
screen  in  their  office  from  which  they  could  operate  the  entire  plant"  (p.  283). 

In  another  cited  study,  Robert  Howard  finds  that  information  systems  "are 
indeed  being  used  to  reproduce  the  logic  of  scientific  management — top  down 
control,  centralization  of  knowledge,  de-skilling— more  comprehensively  than 
ever  before.  (Howard,  1985  p.  207) 

A  skilled  warrior  must  be  fast  (efficient)  and  adaptable  (robust).  It  is 
easy  to  see  how  technologies  can  be  utilized  to  increase  speed  and  efficiency  in 
data  handling — but  it  is  less  clear  whether  this  increase  in  data  availability 
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translates  into  a  smarter  warrior.  A  danger  of  automation  is  that  the  warrior  will 
become  an  automaton  (see  Figure  5).  Thus,  the  system  will  be  fast,  but  brittle. 
The  increased  efficiency  will  be  at  the  expense  of  robustness.  This  is  the 
fundamental  design  challenge — to  harness  the  efficiency  that  information 
technologies  offer  without  sacrificing  the  adaptability  that  the  experienced 
human  can  contribute.  The  objective  is  to  integrate  the  humans  and  technologies 
into  a  system  that  functions  more  efficiently  and  reliably  than  either  would 
function  alone.  The  technology  should  complement  and  amplify  the  human's 
natural  capacities  for  expertise. 

One  of  the  most  exciting  aspects  of  the  relation  between  CSE  and  IW  are 
the  many  analogs  between  the  problem  space  of  the  IW  analyst  and  of  the  CSE 
designer.  Both  the  IW  and  the  CSE  communities  are  faced  with  "messy"  or  "ill- 
defined"  problems.  To  illustrate  that,  we  have  paraphrased  the  characterization 
of  war  used  to  open  this  section  in  terms  of  design: 

Design  is  a  complex  and  chaotic  human  endeavor.  Human  frailty  and 
irrationality  shape  design's  nature.  Uncertainty  and  unpredictability — what 
many  call  the  "fog"  of  design— combine  with  economic  risks,  political 
pressures,  and  human  frailty  to  produce  "friction,"  a  phenomenon  that  makes 
apparently  simple  problems  unexpectedly,  and  sometimes  insurmountably, 
difficult.  Uncertainty,  unpredictability,  and  unreliability  are  always  present, 
but  sound  doctrine,  leadership,  organization,  core  personal  values, 
technologies,  and  training  can  lessen  their  effects. 

In  some  respects,  IW  is  not  about  a  type  of  war  that  will  be  fought  in  the 
future  against  a  to-be-named  enemy.  IW  is  a  battle  that  is  raging  at  this  instant. 

It  is  a  war  between  humans  and  technology.  It  is  a  struggle  to  see  whose  "will" 
will  dominate.  Will  advances  in  information  technology  be  instruments  for 
furthering  human  intentions,  or  will  humans  be  overwhelmed  by  the  complexity 
and  become  hostages  to  those  technologies?  In  this  respect,  CSE  is  an  offensive 
(proactive)  strategy  for  fighting  this  war.  As  Section  II  illustrates,  this  strategy  is 
a  natural  evolution  that  stems  from  struggles  with  earlier  technologies.  Section 
m  presents  a  general  framework  for  building  a  Common  Battlespace  Picture  for 
planning  this  war.  Section  IV  begins  to  flesh  out  this  Common  Battlespace 
Picture  relative  to  a  preliminary  reconnaissance  of  the  battlespace.  Section  V 
suggests  some  general  tactics  and  some  specific  first  steps  for  engaging  the 
enemy. 


The  goal  is  to  impose  our  will  on  complex  technologies.  We  would  like 
to  develop  a  unified  approach  that  is  simple,  economical,  and  has  clearly  defined 
objectives.  We  would  like  to  be  proactive  in  attacking  this  battlespace — we  want 
to  dictate  the  pace  of  the  engagement  and  we  want  to  mass  our  resources  where 
they  will  have  the  greatest  positive  effect  with  the  minimum  risks.  In  many 
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respects,  technology  is  the  barbarian  at  our  gate  and  CSE  represents  our  best 
strategy  for  preparing  for  victory  by  design.  Victory  over  our  information 
technologies  may  be  the  most  effective  path  to  peace  in  our  world. 

To  be  prepared  for  war  is  one  of  the  most  effectual  means  of  preserving  peace. 

George  Washington 
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TERMS  &  ACRONYMS 


AW  ACS 

Airborne  Warning  and  Control  System 

BHEP 

Basic  Human  Error  Probability 

CSCW 

Computer  Supported  Cooperative  Work 

CSE 

Cognitive  Systems  Engineering 

DoD 

Department  of  Defense 

ECM 

Electronic  Countermeasures 

ELINT 

Electronics  Intelligence 

ESM 

Electronic  Support  Measures 

GDSS 

Group  Decision  Support  Systems 

GPS 

Global  Positioning  System 

JSTARS 

Joint  Surveillance  and  Target  Attack  Radar  Syste: 

IADS 

Integrated  Air  Defense  System 

IW 

Information  Warfare 

JSEAD 

Joint  Suppression  of  Enemy  Air  Defenses 

OODA 

Observe,  Orient,  Decide,  Act 

PSF 

Performance  Shaping  Factors 

RMA 

Revolution  in  Military  Affairs 

RPD 

Recognition-Primed  Decisions 

SAM 

Surface-to-Air  Missile 

SEAD 

Suppression  of  Enemy  Air  Defenses 

Scud 

Surface-to-Surface  Missile  System 

THERP 

Technique  for  Human  Error  Rate  Prediction 

UAV 

Uninhabited  Aerial  Vehicle 

UCAV 

Uninhabited  Combat  Air  Vehicle 

VR 

Virtual  Reality 
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